Category Archives: Notes

General Notes

GDPR Compliance for event plugins

amr events, amr ical events list, Feeds, Ical RFC5545 / 2445

This statement is for the wordpress plugins amr-events, amr-ical-events-list  and amr events attendees at https://icalevents.com/plugins_downloads/

Briefly: the amr-events plugin creates events as wordpress custom posts. No additional user data is created or stored. The post author (display name and email) is assumed to be the event organiser for purposes of complying with the ics feed RFC5545 standard. The plugins do not send or store data on any other servers.


Collecting & storing of user data:


The amr-events plugin creates events as wordpress posts.  As per standard wp processing the author of the post is recorded.  That is the extent of any collecting any personal user data that is within the plugins control – only as per standard wordpress user functioning.   The plugin does not of itself collect and store user data.  Control of that user data rests with the website owner.


Processing or use of user data (email and display name):

In event feeds (ics feed) as per the RFC5545 standard the event organizer must be included in the RFC 5545 ics file.  Refer https://tools.ietf.org/html/rfc5545#section-3.8.4.3   – see Organizer

This property MUST be specified in an iCalendar object
      that specifies a group-scheduled calendar entity.  This property
      MUST be specified in an iCalendar object that specifies the
      publication of a calendar user's busy time.

To conform to this, the event post author’s data is then included in the ics file as per the example in the specification :

ORGANIZER;CN=John Smith:mailto:jsmith@example.com

The plugin uses the wordpress post authors “display name” if it exists, if not then the wordpress users “userlogin”, and then the wordpress users ‘user_email’ to provide the required ORGANIZER details.  The plugin does not have any means to validate this data and simply uses what has been stored the user records by the website.

A web developer can choose to display (or not) the public ics url.  The web developer can also control whether a real user or a dummy user is recorded as the post author.

The plugins can also list events from external public ics files.  These ics files may contain attendee data.  The ics files are cached and refreshed daily or as requested.  The cache purpose is to continue to be able to list events should there be a disruption in access to source url.   If the web designer chooses to use the addon https://icalevents.com/downloads/amr-events-attendees/  then the web designer can choose to display that attendee data in the event list.  That list may be displayed in a  public or private page as setup by the web designer.  There is currently no provision to add attendee data to the custom events created inside wordpress.  If this becomes available, a link to the user data such as the user id or email address is what would be stored.

GDPR Summary

The plugins use a minimal amount of user data required to provide event functionality in adherence with the ‘purpose’ and ‘data minimisation’ principles.  With respect to the other GDPR principles, the plugins do not control the accuracy, storage, or security of this user data.   The plugin does not pass user data on to any other parties except as required by the RFC5545 ics specification and as configured by the website developer.

Very useful google calendar toolkit – GcalToolKit

Notes

There are tools that I am very grateful that someone created them.  They are useful and do their specific job well. Things like notepad++, and if you work with events – the GcalToolKit:  If you work with multiple google calendars and use calendar ics files frequently, then you may well find yourself one day in a spot of bother.

Google doesn’t make it easy to bulk manage events, and if your calendar provider has not yet figured out that offering calendar subscriptions is the intelligent way to go.  I have a certain provider of school management software in mind, who several years ago said they were ‘working on this feature’, still don’t have it.  This means that each term (!!!) and every time my child’s timetable changes, I have to logon and redownload the ics file, delete existing events and re-import. And no, they don’t do recurring events – each class is a separate entry (shock horror)

So today I want to give  many thanks to  Neil Gerstenberg for his Google calendar toolkit.

Google Calendar Events Bulk Delete and Management

Correct Handling of UID, RECURRENCE-ID, SEQUENCE

amr events, amr ical events list, Ical RFC5545 / 2445, Notes

Modifications to events, especially to an instance of a recurring event, or an update to a modified instance can all seem quite complex.  It’s not that bad once you understand the various bits that you need to provide for the receiving application to make sense of it. See the RFC5545 spec for details.  This tools.ietf.org version or icalendar.org or the kanzaki version.

  • UID – must have and must be unique
  • SEQUENCE – numerically update to indicate which is the latest version
  • RECURRENCE-ID – if modifying a single instance with in a recurring stream.  Note that for a UID and SEQUENCE pair, the “RECURRENCE-ID” value for a recurrence instance is fixed. The event date may change but the “RECURRENCE-ID” does not. It must always be the original date as generated by the RRULE.
  • LAST-MODIFIED – Probably a good idea top update this too

Note also:

  • if a user subscribes to an ICS URL, then the receiving application will check for updates on a regular basis at a frequency determined by it.  It will be seen as a separate calendar.  It will use the SEQUENCE numbers to determine the latest event details. There may be a delay in an update being made on the originating system and it being reflected in the receiving system.   The only reliable way to be sure that users will have updates seen in their calendars is if you can get them to subscribe to the url (or ‘import’ the url, not the file)
  • if an ics file is imported (eg: when event details are sent as an ics file attachment, not as a url, or if the user downloads the ics file from a url and then imports it), then it will be added as events into the calendar chosen by the user.  Updates may overwrite IF the user re-imports the events AND the application is clever enough to match up the UID’s (and RECURRENCE-IDs if it’s part of a recurring set).  It should then also use the SEQUENCE to determine if the details are actually the latest.  There is no way on devices to force a user to load an updated ics file sent to them.

Example

Modifications to recurring rules or rdates and modifications to instances of those recurring rules (themselves already a modification.)

Assume we have a recurring event.  It starts with a SEQUENCE: 0

DTSTART;VALUE=DATE:20140421
 RRULE:FREQ=MONTHLY;INTERVAL=1;BYDAY=3MO
 UID:1f4irrb5uh6q3cdfsnq15vd00g_R20140421@google.com
 SEQUENCE:0

This generates an infinite set of mondays, including the following

  • 19th May
  • 16th June
  • 21 July
  • 18 August
  • etc

Now, one of the instances gets modified.  (This could be just the description or the actual date.)  A RECURRENCE-ID VEVENT is created for each instance modification and the SEQUENCE NO is updated.   The recurrence-id matches the date in the recurring rule that it applies to. The combination of UID, RECURRENCE-ID and SEQUENCE helps us identify the instance that is being updated.

For a given pair of "UID" and "SEQUENCE" property values,
   the "RECURRENCE-ID" value for a recurrence instance is fixed.
DTSTART;VALUE=DATE:20140721  (ie no date change in this example)
 DTSTAMP:20140720T064853Z
 UID:1f4irrb5uh6q3cdfsnq15vd00g_R20140421@google.com
 RECURRENCE-ID;VALUE=DATE:20140721

Now assume the original RRULE is modified too, perhaps just the description or shock horror, the recurring date definition?  It’s sequence-no is updated too BUT now what should happen to the RECURRENCE-ID instance?  Should it be updated to indicate it is still valid?  IE we now have:

  • generated date 21 July 2014 with UID and SEQUENCE = 3 and
  • RECURRENCE-ID;VALUE=DATE:20140721 with matching UID and SEQUENCE = 3

It appears that perhaps SEQUENCE numbers can increment separately, so one could conceivably receive a file with a RRULE with an equal to or  greater sequence number than a RECURRENCE-ID.       From   this discussion, it is really not clear what the supplying calendar application will do:

Once UID 1/SEQUENCE 0 hits UID 1/SEQUENCE 1 (which would
mean something significant about the set of instances
has changed) all bets are off in regards to RECURRENCE-ID
values that previously existed.

If the UID/RECURRENCE-ID/SEQUENCE matches a generated date from the rrule with same SEQUENCE (ie both have been modified), then what?  One cannot just use eg: LAST-MODIFIED:20140620T200327Z.  Who is to say that a mod of the RRULE was intended to overwrite the RECURRENCE instance?   Quite possibly not.

Some lengthy related discussions here and in other parts of the thread:

Complications that urge one to keep events as simple as possible:

Searching the web seems to indicate some folks believe RECURRENCE-IDS should not change.  So should we assume that if the RECURRENCE-ID vevent is still in the file then it is still valid and should be included.  (What does this mean for syncing and deletions?  IE How does one ‘delete’ an existing event.  Or should one always delete events from an ics file and reload ONLY what’s in the ics file?)

Dates, Times, Formatting

amr events, amr ical events list, Ical RFC5545 / 2445,

There are a number of date and time options for you to choose when displaying your events.    The main ones you should be using are created by the plugin and will be formatted by the date and time settings you have specified in the plugin settings (and applying any localisation that may be available in your wordpress setup).  You can specify these by listtype, so say a widget could have a very compact date format and a full list may be more detailed.

screnshot Date Time Formats
Date Time Format settings by listtype

Event date fields formatted by your date settings:

  • EventDate: Mon, June 23, 2014
  • EndDate: Mon, June 30, 2014 (only if event greater than one day)

Event time fields formatted by your time settings:

  • StartTime: 9:00 am
  • EndTime: 10:00 am

Note there may be situations where there is no end time or no end date (eg: if there is no duration or the end date is the same as the start date.   If a field has not been populated, it will not be displayed and neither will any ‘before’ or ‘after’ text.

Field from the ics specification that may be useful.

Formatted using human language function

  • DURATION:1 week , 1 hour

Other date fields in the ‘edit events’ admin screen

screenshot of show screen options in edit events
show screen options in edit events

  • Last 20140630

The ‘last’ date shown in the edit events admin screen shows the last recurrence date of an event.  It is used to select events within a date range (ie exclude events whose recurrences are in the past, without having to rebuild the recurrence logic).  It is also used to sort events in the admin screen by this last recurrence.   It is therefore formatted yyyymmdd for the sorting.   It is not intended to be displayed elsewhere.

Technical fields

These are direct from the ics specification formatted by the DateTime settings  in the plugin settings.   Note the DTEND has a special definition in the ics specification for all day events (it will have the next day as the end date) and may be confusing to normal users.   Some ics generators get the DTEND logic wrong, so this plugin tries to work with duration just in case.

  • DTEND: Mon, June 30, 2014 10:00 am
  • DTSTART: Mon, June 23, 2014 9:00 am

Other fields

As these are very technical, precise fields, they are by default not formatted but left as found.

  • DTSTAMP: 20140616T021842Z
    • the date and time that the instance of the iCalendar object was created.
  • LAST-MODIFIED:20140616T025756Z
    • the date and time last modified – works with the sequence number to determine the latest modification to an event instance.

Google Calendar, Timezones and daylight saving

Notes, , , ,

I’m fully aware of how complicated and dangerous working with timezones can be, especially if recurring events and daylight saving are involved.

Outsourcing calendar event creation

So when I decided that I wanted my kids school timetables in ics format and that I’d ‘outsource’ the manual data entry of it, I took great care to setup the calendars myself, specifying the Australia/Sydney Timezone.     I found a great person on elance based in the states who was knowledgeable about google calendar and recurring events.  I shared the time table calendars with her.  She set up the events in the two timetables. I checked it and all was good for a few weeks.

Then the normal Sydney (NSW) daylight saving period ended.

And NOW all the timetable events were TWO hours out – yes two (2) ?? why?

Being knowledgeable in these matters and having done SO many tests with timezones changing daylight saving at different overlapping times etc, I knew that it must be something to do with timezones.  I logged on to google and could not find anything online that told me anything.  Timetable calendars still set to Sydney, but all the events now mysteriously 2 hours out.

Troubleshooting google calendar – check the ics file

I downloaded the ics file to see if that would shed any light.    The ics file had a timezone definition for Australia/Sydney AND for America/New_York and ALL the class events were defined with timezone America/New_York.

Now google very clearly says that it stores the events in UTC / GMT time.

So why export events for a timezone OTHER than

  • either UTC or the
  • calendars timezone or
  • the accounts timezone?

why not just one hour out?     Had I missed the New York daylight saving change on 9 Mar?  Maybe ?   The times are just of the hour like 10.50 instead of 8.50 – maybe when I saw 9.50 I just thought 9 ?  who knows?

How to avoid unintended timezone daylight saving clashes happening again ?

Well I’m guessing – it appears google will use the timezone of the person creating the event EVEN if the calendar is in a different timezone AND this timezone is NOT shown when one looks at the event, NOT even in event debug / troubleshooting mode (add &eventdeb=1 to your google calendar url).

The creator has to manually assign a timezone to each and every event to ensure that it gets exported correctly.

 How to fix the current timetables offset by two hours?

I didn’t want to delete the existing calendars and Google allows no other easy way to delete events from any non-main calendar other than deleting the whole calendar.

I decided  a temporary fix (until the next daylight change !)  would be to export to ics, do a global change to a timezone 2 hours before New_York (hopefully one with out daylight saving – sadly I gave up on that)

I created a test calendar and did several iterations to check the timings. I ended up using http://gcaltoolkit.net/ to handle bulk deletes of the calendars and learnt a few things along the way:

For example: Even if you have deleted the events and want to re-import them, google will not let you.  To fox it – do a gloabl change on the @google.com of the UID .  Events are identified by UID.

At one stage I even felt I needed to change the file name and calendar name, but perhaps I was just tired, confused and annoyed by then.

Summary:

  1. BACKUP all your google calendars and possibly other data first before letting another app mess with your data.  Create a TEST calendar to test the whole process.
  2. Export your ics files, create a backup copy.
  3. Use something like notepad++ to do a search/replace on the timezone.  Make sure your new timezone is something google will recognise and the correct UTC offset.
  4. Search and replace the @google.com to @yourdomain.com or really anything…(alter the event UID globally every time you re-import the file.)
  5. Save and close the ics file.
  6. Use gcaltoolkit or similar to delete the events in your target calendar (very carefully check which calendar you are accessing)
  7. Make a cup of tea – ieWait a few minutes, sometimes it takes google a while to update.
  8. Import the ics file and check the times.

 

 

 

Google indexing, bandwidth and calendar parameters

Notes, , , ,

Dilemma

spiders clogging up the works
March 2012 in Australia has been very wet - the spiders don't like it any more than we do!

Summary

The events plugin does what it can to help get your events indexed (see below).  Dynamic event lists change frequently (daily or more?) as time goes by – they drop off old events and show the newer ones.  These frequent “updates” could perhaps cause more attention from a search engine.    Widgets (box and calendar) that have internal links back to the main calendar page can also make google think that page is important.  Especially if those widgets are on every page.

Maybe that is what you want, and maybe it is not!  You may need to control it.

You the webmaster can help the seach engine spiders or crawlers further to index your site while not chomping all your bandwith.

How events plugin helps indexing:

Other tools to manage indexing and bandwidth

How to get your events indexed:

How to manage bandwidth

Notes

  • Think carefully about using multple views and about the sidebar widgets that can generate a plethora of links back to the calendar page -Get other measures  in place to avoid overemphasising the calendar page importance
  • Attempting to avoid bandwith issues may affect indexing – wrt query string parameters.  Best is to ‘tell’ google what to focus on.  See webmaster guidelines
  • It seems google will be indexing the POST variables too and they advise sticking with GET (ie the query parameters)
  • Hmm – maybe adding rel=”next” and “prev” to pagination will help avoid diluting page rank? See googles video or article
  • Also we should consider sidebar widget issues – links on every page that point to the same  may overly highlight the calendar page and make it more important – sometimes this is intentional and sometimes not.  Related question via Brad Gosse: 2010/12/06/do-tag-clouds-in-sidebars-affect-seo-ranks?    Should the widgets have no follow links on certain links ? …. but we want that indexing….maybe just the rel control?

Html in descriptions in ics feed generation

amr events, Html, Ical RFC5545 / 2445, ,

What to do about html in event descriptions?

Some applications will cope with the html, but there are other applications which do not. EG: Google calendar – does not faciliate html nor  issue it in feed.

HTML in the description field is not explicitly forbidden, however it is probably not recommended.  There is some provision for html to be explicitly allowed for, either:

  •  in the description using an ALTREP parameter,
  • or in an ‘experiemental’  X field, X-ALT-DESC,

WordPress allows us to create events with html in them.  So what to do when the ics feed is generated?

Retaining the html – two possibilties:

Link to the html representation:

DESCRIPTION;ALTREP="http://www.example.com/somepath/":This is an example description.

Provide separate field with html representation:

Outlook will recognise:

X-ALT-DESC;FMTTYPE=text/html:<p>This is an example description.</p>

 

This update does both, as well of course ,as stripping the html from the description field in the ics file.  This means images will be stripped too (only for the feed – do not panic!)

More information for the technically minded

 

 

Problem with Dates/Time – Off by 1 Hour?

amr events, amr ical events list, Ical RFC5545 / 2445, ,

If you are using amr-events or amr-ical-events-list you will not have this problem!

Every time a zone somewhere in the world kicks on to daylight saving or off for that matter, I see a bunch of support questions popping up about their event times being off by an hour.    (I monitor wordpress support, stackexchange and wpquestions, and maybe a few more…..)

From a favourite cartoon source - thanks http://xkcd.com/673/

Imagine the confusion this can cause where clients may have subscribed to calendar feeds and have the wrong times.

PHP datetime objects and functions

If people are very lucky, the plugin that they are using has some good coding and is using php DateTime objects with timezone support, and the only problem is that their wordpress timezone is set as a GMT offset, not as a timezone.

That is easily fixed! (goto wordpress general settings)

However all too often the plugin developer has failed to appreciate

  • the complexities of times around the world,
  • timezones,
  • daylight saving and
  • most importance of managing recurring events in one timezone for a display in another where one zone changes daylight saving.  Very easy to generate the wrong times for a recurring event without realising it!
  • even worse governments sometimes change the date/time of the switchover (ok not that often, but it does happen, and then you hope your php installation gets updated with the timezone definitions)

Fixes:

If it is not a simple wordpress timezone setting, then these things are NOT easy to fix and will involve changes to code:

  1. Change to use php DateTime Classes for all date logic .If they are using unix time and not  php datetime objects, it is almost impossible to get it right.  It’s a fairly major rewrite to change all date logic to use the DateTime objects and functions.
  2. If already using, it maybe that the intricacies of recurring event logic have won out.  See below for logic help.

Developers and event website managers should read:

 

 

WordPress events plugin with xml output? want it? why?

Feeds,

The IETF (Internet Engineering Task Force) has just published the RFC (Request for Comment) for xcal (iCalendar in XML).

Given that the ics format works fairly well to give us calendar exchange and interoperability (given the natural constraints of human involvement in import/subscribe to their calendar apps) , does one need an xml format too ?

An argument for the existence of an xml standard is that people have already developed their own xml versions – a standard was required for interchange of data.

If you have thoughts on this and would like to see that this wordpress events plugin produce an xml icalendar feed as well as the ics feed, please add  a comment below.

Try to be as specific as possible as to the reasons and benefits to motivate the addition of the xml format – export (and hmmm maybe import).

Some input to get you going:

I think the biggest software revolution of the future is that the calendar will be the organizing filter for most of the information flowing into your life. You think you are bombarded with too much information every day, but in reality it is just the timing of the information that is wrong. Once the calendar becomes the organizing paradigm and filter, it won’t seem as if there is so much.

 

 

hcalendar microformats By the way, did you know this plugin already issues the necessary css classes for the hcalendar microformat?

 

 

Ical Events and default URLs for the widget

Html, Notes

Event Description as href title - hover text
Event Description as href title – hover text

 Using amr-events with internal events?

  • Your events all have a link to their post or custom post type.
  • You can switch the hover text off (see widget settings)

Using amr-ical-events-list (or an external ics file)?

In an ics file sometimes (quite often?)  there is no event URL.  To deal with this, you have the following:

  • one can specify a default url (such as a calendar page) , so that if people click on the event in the widget, they go to the default event url.  (See below for configuration)
    • either for the list type
    • or in the widget settings

Configuration of the Default Event URL for the widget events

 

Specify the List Type to use for the Widget
Specify the List Type to use for the Widget

 

Default event url in listtype settings