Correct Handling of UID, RECURRENCE-ID, SEQUENCE

Modifications to events, especially to an instance of a recurring event, or an update to a modified instance can all seem quite complex.  It’s not that bad once you understand the various bits that you need to provide for the receiving application to make sense of it. See the RFC5545 spec for details.  This tools.ietf.org version or icalendar.org or the kanzaki version.

  • UID – must have and must be unique
  • SEQUENCE – numerically update to indicate which is the latest version
  • RECURRENCE-ID – if modifying a single instance with in a recurring stream.  Note that for a UID and SEQUENCE pair, the “RECURRENCE-ID” value for a recurrence instance is fixed. The event date may change but the “RECURRENCE-ID” does not. It must always be the original date as generated by the RRULE.
  • LAST-MODIFIED – Probably a good idea top update this too

Note also:

  • if a user subscribes to an ICS URL, then the receiving application will check for updates on a regular basis at a frequency determined by it.  It will be seen as a separate calendar.  It will use the SEQUENCE numbers to determine the latest event details. There may be a delay in an update being made on the originating system and it being reflected in the receiving system.   The only reliable way to be sure that users will have updates seen in their calendars is if you can get them to subscribe to the url (or ‘import’ the url, not the file)
  • if an ics file is imported (eg: when event details are sent as an ics file attachment, not as a url, or if the user downloads the ics file from a url and then imports it), then it will be added as events into the calendar chosen by the user.  Updates may overwrite IF the user re-imports the events AND the application is clever enough to match up the UID’s (and RECURRENCE-IDs if it’s part of a recurring set).  It should then also use the SEQUENCE to determine if the details are actually the latest.  There is no way on devices to force a user to load an updated ics file sent to them.

Example

Modifications to recurring rules or rdates and modifications to instances of those recurring rules (themselves already a modification.)

Assume we have a recurring event.  It starts with a SEQUENCE: 0

DTSTART;VALUE=DATE:20140421
 RRULE:FREQ=MONTHLY;INTERVAL=1;BYDAY=3MO
 UID:1f4irrb5uh6q3cdfsnq15vd00g_R20140421@google.com
 SEQUENCE:0

This generates an infinite set of mondays, including the following

  • 19th May
  • 16th June
  • 21 July
  • 18 August
  • etc

Now, one of the instances gets modified.  (This could be just the description or the actual date.)  A RECURRENCE-ID VEVENT is created for each instance modification and the SEQUENCE NO is updated.   The recurrence-id matches the date in the recurring rule that it applies to. The combination of UID, RECURRENCE-ID and SEQUENCE helps us identify the instance that is being updated.

For a given pair of "UID" and "SEQUENCE" property values,
   the "RECURRENCE-ID" value for a recurrence instance is fixed.
DTSTART;VALUE=DATE:20140721  (ie no date change in this example)
 DTSTAMP:20140720T064853Z
 UID:1f4irrb5uh6q3cdfsnq15vd00g_R20140421@google.com
 RECURRENCE-ID;VALUE=DATE:20140721

Now assume the original RRULE is modified too, perhaps just the description or shock horror, the recurring date definition?  It’s sequence-no is updated too BUT now what should happen to the RECURRENCE-ID instance?  Should it be updated to indicate it is still valid?  IE we now have:

  • generated date 21 July 2014 with UID and SEQUENCE = 3 and
  • RECURRENCE-ID;VALUE=DATE:20140721 with matching UID and SEQUENCE = 3

It appears that perhaps SEQUENCE numbers can increment separately, so one could conceivably receive a file with a RRULE with an equal to or  greater sequence number than a RECURRENCE-ID.       From   this discussion, it is really not clear what the supplying calendar application will do:

Once UID 1/SEQUENCE 0 hits UID 1/SEQUENCE 1 (which would
mean something significant about the set of instances
has changed) all bets are off in regards to RECURRENCE-ID
values that previously existed.

If the UID/RECURRENCE-ID/SEQUENCE matches a generated date from the rrule with same SEQUENCE (ie both have been modified), then what?  One cannot just use eg: LAST-MODIFIED:20140620T200327Z.  Who is to say that a mod of the RRULE was intended to overwrite the RECURRENCE instance?   Quite possibly not.

Some lengthy related discussions here and in other parts of the thread:

Complications that urge one to keep events as simple as possible:

Searching the web seems to indicate some folks believe RECURRENCE-IDS should not change.  So should we assume that if the RECURRENCE-ID vevent is still in the file then it is still valid and should be included.  (What does this mean for syncing and deletions?  IE How does one ‘delete’ an existing event.  Or should one always delete events from an ics file and reload ONLY what’s in the ics file?)

Dates, Times, Formatting

There are a number of date and time options for you to choose when displaying your events.    The main ones you should be using are created by the plugin and will be formatted by the date and time settings you have specified in the plugin settings (and applying any localisation that may be available in your wordpress setup).  You can specify these by listtype, so say a widget could have a very compact date format and a full list may be more detailed.

screnshot Date Time Formats
Date Time Format settings by listtype

Event date fields formatted by your date settings:

  • EventDate: Mon, June 23, 2014
  • EndDate: Mon, June 30, 2014 (only if event greater than one day)

Event time fields formatted by your time settings:

  • StartTime: 9:00 am
  • EndTime: 10:00 am

Note there may be situations where there is no end time or no end date (eg: if there is no duration or the end date is the same as the start date.   If a field has not been populated, it will not be displayed and neither will any ‘before’ or ‘after’ text.

Field from the ics specification that may be useful.

Formatted using human language function

  • DURATION:1 week , 1 hour

Other date fields in the ‘edit events’ admin screen

screenshot of show screen options in edit events
show screen options in edit events
  • Last 20140630

The ‘last’ date shown in the edit events admin screen shows the last recurrence date of an event.  It is used to select events within a date range (ie exclude events whose recurrences are in the past, without having to rebuild the recurrence logic).  It is also used to sort events in the admin screen by this last recurrence.   It is therefore formatted yyyymmdd for the sorting.   It is not intended to be displayed elsewhere.

Technical fields

These are direct from the ics specification formatted by the DateTime settings  in the plugin settings.   Note the DTEND has a special definition in the ics specification for all day events (it will have the next day as the end date) and may be confusing to normal users.   Some ics generators get the DTEND logic wrong, so this plugin tries to work with duration just in case.

  • DTEND: Mon, June 30, 2014 10:00 am
  • DTSTART: Mon, June 23, 2014 9:00 am

Other fields

As these are very technical, precise fields, they are by default not formatted but left as found.

  • DTSTAMP: 20140616T021842Z
    • the date and time that the instance of the iCalendar object was created.
  • LAST-MODIFIED:20140616T025756Z
    • the date and time last modified – works with the sequence number to determine the latest modification to an event instance.

Complete multi-lingual event website

To achieve a complete multi-lingual website using amr-events and amr-ical-events-list, the following are required:

the wordpress language files

These are required for the day, month etc event time localisations.  See this post for instructions and links.

your theme language files

Some theme language files can be found at http://i18n.svn.wordpress.org/
. You may have to click down various paths as sometimes somelanguages are stored differently.
Danish example: http://i18n.svn.wordpress.org/da_DK/trunk/messages/twentythirteen/

Updated plugin language files

Some folks have kindly provided translations.  You can see the latest available here.   These are not always up to date.  See translation tools below.  If you need to update your language files, please send me the update and I’ll upload it.

If you do NOT send an update, your version may be overwritten in a plugin update unless you move it.

Custom Language File Location

For these two plugins, you can store your custom .mo and .po files in your WP-CONTENT languages subfolder.

Do not use the subfolder /plugins as wordpresss may one day be auto loading translations to that folder.

custom_language_file

Translation tools:

Finally a Language switcher and/or a multi-lingual language plugin. Some plugins that may help:

  • Xili (unverified in recent years)

Demonstration

lang.icalevents.com

Google Calendar, Timezones and daylight saving

I’m fully aware of how complicated and dangerous working with timezones can be, especially if recurring events and daylight saving are involved.

Outsourcing calendar event creation

So when I decided that I wanted my kids school timetables in ics format and that I’d ‘outsource’ the manual data entry of it, I took great care to setup the calendars myself, specifying the Australia/Sydney Timezone.     I found a great person on elance based in the states who was knowledgeable about google calendar and recurring events.  I shared the time table calendars with her.  She set up the events in the two timetables. I checked it and all was good for a few weeks.

Then the normal Sydney (NSW) daylight saving period ended.

And NOW all the timetable events were TWO hours out – yes two (2) ?? why?

Being knowledgeable in these matters and having done SO many tests with timezones changing daylight saving at different overlapping times etc, I knew that it must be something to do with timezones.  I logged on to google and could not find anything online that told me anything.  Timetable calendars still set to Sydney, but all the events now mysteriously 2 hours out.

Troubleshooting google calendar – check the ics file

I downloaded the ics file to see if that would shed any light.    The ics file had a timezone definition for Australia/Sydney AND for America/New_York and ALL the class events were defined with timezone America/New_York.

Now google very clearly says that it stores the events in UTC / GMT time.

So why export events for a timezone OTHER than

  • either UTC or the
  • calendars timezone or
  • the accounts timezone?

why not just one hour out?     Had I missed the New York daylight saving change on 9 Mar?  Maybe ?   The times are just of the hour like 10.50 instead of 8.50 – maybe when I saw 9.50 I just thought 9 ?  who knows?

How to avoid unintended timezone daylight saving clashes happening again ?

Well I’m guessing – it appears google will use the timezone of the person creating the event EVEN if the calendar is in a different timezone AND this timezone is NOT shown when one looks at the event, NOT even in event debug / troubleshooting mode (add &eventdeb=1 to your google calendar url).

The creator has to manually assign a timezone to each and every event to ensure that it gets exported correctly.

 How to fix the current timetables offset by two hours?

I didn’t want to delete the existing calendars and Google allows no other easy way to delete events from any non-main calendar other than deleting the whole calendar.

I decided  a temporary fix (until the next daylight change !)  would be to export to ics, do a global change to a timezone 2 hours before New_York (hopefully one with out daylight saving – sadly I gave up on that)

I created a test calendar and did several iterations to check the timings. I ended up using http://gcaltoolkit.net/ to handle bulk deletes of the calendars and learnt a few things along the way:

For example: Even if you have deleted the events and want to re-import them, google will not let you.  To fox it – do a gloabl change on the @google.com of the UID .  Events are identified by UID.

At one stage I even felt I needed to change the file name and calendar name, but perhaps I was just tired, confused and annoyed by then.

Summary:

  1. BACKUP all your google calendars and possibly other data first before letting another app mess with your data.  Create a TEST calendar to test the whole process.
  2. Export your ics files, create a backup copy.
  3. Use something like notepad++ to do a search/replace on the timezone.  Make sure your new timezone is something google will recognise and the correct UTC offset.
  4. Search and replace the @google.com to @yourdomain.com or really anything…(alter the event UID globally every time you re-import the file.)
  5. Save and close the ics file.
  6. Use gcaltoolkit or similar to delete the events in your target calendar (very carefully check which calendar you are accessing)
  7. Make a cup of tea – ieWait a few minutes, sometimes it takes google a while to update.
  8. Import the ics file and check the times.

 

 

 

Importing events from a csv file

Simple ‘flat’ events can be imported from a csv file.  This is possible with a third party csv import plugins. The amr-events plugin attempts to use the wordpress meta data as simply as possible to facilitate this kind of easy integration with other plugins.

Events can be complicated: they can recur, be in different timezones, have exceptions to their recurring definition, have alterations to some instances etc.    These ‘complications’ cannot so easily be imported.

From a brief review of the csv import plugins out there, starting with the self named “ultimate csv importer” and backing away in disappointment, I read reviews of a few others and decided to try the really simple importer.   It worked so nicely and cleanly that I stopped there!

What a pleasure to see a plugin that

  • inserts itself sensibly into wordpress, adding itself under tools > import just where it should be
  • imports the data with limited fuss, very sensibly using the column headings, making sensible defaults for missing fields instead of insisting that they are mandatory data. eg: choose to include a post_status, or let it default to draft and ‘publish’ it later.
  • handles custom fields and custom post types as easily as they should be handled – why should there be a ‘fuss’

 So how to import events?

I suggest you create yourself a test event or two that are similar to the event data that you wish to import.  This is so that you can see the meta data that is created.  The amr events plugin tries to stay close to the ICS RFC spec formats for the data, but is sophisticated enough to cope with a reasonable set of date formats and will work with minimal event data input.   Sample files provided below.   The plugin needs

  • some of the standard wordpress post data, as little as your chosen csv importer will cope with.
    • post_title
    • post_content
    • post_type = event (or whatever you have in your settings, or load as post and change afterwards if your csv importer is a bit ‘dumb’.)
  • a minimal set of event meta data (see example file for variations):
    • _DTSTART in RFC format eg: 20140402T090000   or 20140402
  • optional extras:
    • _DTEND  (to define any timed events or events of greater than 1 day.  NOTE RFC 5545 says _DTEND ends on NEXT day for one day events – be careful what you define and test it!)
    • _timezone (if timezone differs from UTC or your website timezone – careful testing with your setup)
    • _allday (timed or allday)
    • _LOCATION
    • see your sample events post-meta in phpmyadmin for more possibilities.

Notes:

  • It is probably not practical to try CSV importing recurring _RRULE data or detailed duration data (use DTEND rather)
  • If you experience any problems, please consider carefully whether it is a problem with your chosen CSV importer and if so, log your support query there.
  • If some data goes in a bit wonky – maybe you set it up a bit wrong, the amr-events plugin does try to correct things – update the post and save.

Screenshots:

imported csv events
variety of events with various data supplied or omited
really simple csv  import screen
really simple csv import screen

 

import csv menu
The csv import option is right there in the wordpress import menu – exactly the right place

Example Event CSV file:

minimal_events

Styling the dates

I received a question about whether the plugin could style dates in the way the little purple date is shown in the header image of the website and the plugins wordpress page.

a real life example
a real life example

 

The calendars shown in the header are taken from real live examples of the plugin being used in various sites.  The sites will in many cases have added custom css to style the events lists to match their themes.

With a combination of the date and time formats in the settings for each list and some css, almost anything is possible.

The key is that one can add extra characters into the php date format string.  If the characters are the same as a date format character, one can ‘escape’ them with a preceding backslash.

Sooo…. one could for example add some html to isolate part of the date for css styling purposes.   The image below shows the date and time format settings for a event list.   Note the format string for the day.   It surrounds the month with a span.

\<\s\p\a\n\>M\<\/\s\p\a\n\>j
adding html to the date format
Date format settings for a event listtype

Then one can spice it up with some custom css.  To prove this, I have used the power of the firebug browser add-on to add some element css to show you what could be done.  I’d put the text-align: center at a higher level actually – no need to repeat it.

You of course would add the css for the event date class and the span within it to your theme or to a custom calendar css file.

Example styling of a month and day (prof of concept facilitated by firebug browser add-on)
Example styling of a month and day (prof of concept facilitated by firebug browser add-on)

Finding the Facebook events ics feed for subscribing or syncing

I’m not a fan of facebook…..however I get asked the questions, so have had to develop some familiarity, so here’s a quick reference until facebook changes things again.

Before you get too stuck in to facebook for your events,  look at oreilly.com’s advice on perhap not using facebook as the home for your public events.

Finding that public events ics feed in facebook

From a facebook ‘page’

They really do not make it easy do they ?  There is no obvious ‘Export all’ as there is in the personal page.  So…Pretend you’re only exporting one event from your page.

Example fb page
Example Facebook page – events page

Click the event

The facebook events lists, click export.

Click export.

Choose subscribe to all events, or right click and copy the link address

Copy that webcal ics address and use it wherever you need to subscribe to that set of events.

From a personal homepage

After logging in look for the export events option, not just for one event but ALL upcoming events.

Go to your all events page:   https://www.facebook.com/events/list.

Click "Export"In the top right corner, see the little gear icon, click it and click the export option.

 

You should get a popup window.

export facebook ics
export facebook ics

If you just want the url to subscribe to it from another application, right click and ‘copy the link address’. Paste it in to your calendars subscribe url option.

Alternatively if you just click on it, your calendar application should open up and hanlde the subscription process for you.

Use it in the ical events plugin?

facebook in shortcode
facebook in shortcode

Bear in mind that the plugin caches the url’s, so you may need to force a refresh (add ?refresh) to the calendar page url and refresh the page in your browser.

eg:

 

Other possibly useful references:

Table css, overlapping or disappearing sidebars

Table widths and css

Tables are wonderful  – they expand beautifully and naturally to fit your content, but sometimes that same content can make the table do  weird things.

Loooong Urls

Please be sure to test your css in your theme with representative content.

The plugin tries to deal with some of the problems that occur.  For example long urls that can make a table expand too much.   See also plugin-and-theme-css-not-working-well-together.

Sometimes the only way to really sort a table width out is to “fix” the width of the table and/or the table cells.    Now this the plugin has to leave this to you – some people have wide content areas, some not.  Some have lots of info displayed,  others have minimal.

The plugin has no way of knowing what size would work best for you.

I suggest that you use something like firebug to see what your theme is doing (or not doing!) and then google table+css+width for ideas on what might work with your theme.   You know the size of content area that your calendar or agenda list must fit into and perhaps simply setting the width of the table and/or the table cells will be the answer.

Coping with a wide table

A suggestion from 456bereastreet.com

Css3 options:

  • word-wrap: break-word has some browser support
  • word-break: (normal, break-all, keep-all) aimed at dealing with mixed character sets.  Random Break Point.

Resources:

See posts on this site tagged css.