The major new feature of the 1.2 release is support for non-SyncML protocols in general and CalDAV/CardDAV in particular. ActiveSync support is in development.

Changes 1.1.1 -> 1.1.99.5

The major new feature of the 1.2 release is support for non-SyncML protocols in general and CalDAV/CardDAV in particular. ActiveSync support is in development. These protocols are implemented as backends which are combined with other backends by SyncEvolution in a so called "local sync". The GTK sync-ui does not yet support configuring non-SyncML protocols. See the README and man page for more information on how to use the new feature via the command line.

Properties not supported by SyncML servers can now be preserved locally in two-way synchronization (BMC #15030). This depends on information about what properties a SyncML server supports ("CtCap"), which is typically not provided by servers. SyncEvolution contains a copy of that information for Google Contacts (BMC #15029).

Akonadi backend and KWallet support were merged. They are not included yet in syncevolution.org binaries. To use them compile from source.

The configuration format was updated to solve a conceptual problem inherited with the legacy property names: the "type" property had multiple, sometimes conflicting roles. For example, setting the preferred data format for sync with one peer might have changed the backend selection for some other peer (BMC #1023). Now "backend/databaseFormat/syncFormat/forceSyncFormat" replace "type". "type" is still accepted by the command line as alias.

Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. In contrast to earlier, more experimental releases in the 1.2 series, 1.1.99.5 and later automatically migrate configurations. The old configurations will still be available (see "syncevolution --print-configs") but must be renamed manually to use them again under their original names with older SyncEvolution releases.

Other changes:

• a problem with enabling release mode required replacing 1.1.99.5 with a fixed 1.1.99.5a release
• syncevo-http-server was enhanced considerably. See the HTTP server HOWTO
• support NetworkManager API >= 0.9 (BMC #19470)
• Sync mode is recorded when running in SyncML server mode (BMC #2786).
• syncevo-dbus-server automatically stops when some of its libraries are updated and restarts if auto-syncing is on (BMC #14955).
• Using the --sync-property and --source-property command line options is optional, just specifying the property assignment is enough.
• Added support for Buteo, mKCal and QtContacts in MeeGo. Buteo and mKCal were removed again from MeeGo, so the code is obsolete. The QtContacts backend may be still be useful to access items via that API, but for syncing on MeeGo the normal EDS backend is used since MeeGo reverted back to EDS as PIM storage.
• code cleanup and various minor fixes/improvements, see ChangeLog

Source, Installation, Further information

Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources

i386, amd64 and lpia binaries for Debian-based distributions are available via the "unstable" syncevolution.org repository. Add the following entry to your /apt/source.list, then install "syncevolution-evolution":

These binaries include the "sync-ui" GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default).

The same binaries are also available as .tar.gz and .rpm archives in http://downloads.syncevolution.org/syncevolution/evolution. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise.

After installation, follow the getting started steps.

• Adam Williamson: "The continuing state of contact + calendar synchronization suck"
• Matěj Cepl: Synchronization sucks!

As someone who has worked hard on making synchronization suck less I can't let this stand without commenting... in particular not when SyncEvolution is mentioned as not being reliable! ;-)

Nitpicking

First let me point out some factual mistakes in Adam's post: "Maemo's whole synchronization story" has never been based on SyncEvolution. SyncEvolution is an add-on, supported entirely on a volunteer basis by Ove Kaveh on the device. If people do not get help for SyncEvolution in the Maemo forums, then I can't do anything about it because I don't have the time to keep up with everything that is said (or asked) on the web about SyncEvolution. It's up to the Maemo community to help their peers. SyncEvolution on Linux to N900 broken somewhere outside of SyncEvolution (device, Bluetooth stack) and without support by Nokia for their closed-source sync component on the device I don't see much chances to fix it, short of some Bluetooth experts getting involved.

The statements about MeeGo are also incorrect. Evolution Data Server was not the preferred PIM storage for MeeGo 1.2 until recently, so depending on it for CalDAV/CardDAV support was not an option. Adam points to a bug report where I captured some thoughts around the technical aspects. Perhaps this was too brief to be understand without context, but I still think that the arguments and conclusion are valid. More on that below.

What exactly is the complaint?

Adam tried SyncEvolution with Horde and eGroupware. Other people were able to get these combinations working, for example just a few days ago George Runelli with eGroupware. It seems to depend a lot on the exact setup on the server side. I had offered Adam to help with diagnosing his problem with Horde (unexpected slow syncs), but he never replied to my email.

I'm still convinced that the problem is not in SyncEvolution, but rather on the server side, because SyncEvolution works fine with a variety of other SyncML servers (Funambol, Memotoo, Synthesis, Mobical, to name just those that I test with nightly). It is not SyncEvolution's fault that the open source groupwares seem to have less stable SyncML support.

I tried to work with Horde and eGroupware developers a while back when I started with SyncEvolution. I had a hard time getting anyone to reply to my questions and emails, even when contacting the original developers directly. If the situation is different today, then I'd be happy to restart that effort.

I'm not sure what kind of problem Matěj had with SyncEvolution. He doesn't say in his blog post, only that it does not allow him to reliably sync with his server running Zarafa. I'm not surprised. To the best of my knowledge, the two are unable to synchronize against each other by design, because SyncEvolution is based on SyncML and Zarafa on ActiveSync. So is the complaint that SyncEvolution uses an open protocol and not a patent-encumbered proprietary protocol?

Proposed solutions

Adam then continues to suggest that the data synchronization model itself is flawed and should be replaced with client/server model where changes are always stored on the server immediately, as in Evolution's CalDAV and CardDAV backends. This became more clear in an email discussion after he contacted me regarding his blog post.

The key difference is this:

• True synchronization allows offline modification of the data.
• Capable devices by design store a complete copy of the data, without depending on one particular server to remain online.

Adam argued that a client/server model can be combined with caching of items and changes. But then the client/server model becomes synchronization and must deal with the same kind of problems that it was meant to avoid, like conflicts between items on client and server. Adam later said that changes that cannot be stored anymore should simply be discarded. That doesn't sound like a very useful approach, because users won't be able to remember what changes might have been lost and if they do, would most likely be forced to redo them manually. PIM data is more complex then plain text, so the merge strategies that programers know how to use with source code and revision control systems do not apply. Normal users will be even more challenged.

The point about not depending on a central server is important, too. In the SyncML world we have recently seen that ScheduleWorld shut down. No data was lost, because by design all users always had a full copy of their data on their own storage. The same can't be said for all the popular Web 2.0 cloud services...

My devices are not always online, for practical and economical reasons. Therefore I want the ability to make changes while offline and will continue to work on the more capable model.

SyncML

One other aspect is the question whether the data synchronization model itself is flawed, just some protocols implementing it, or only specific implementations of these protocols.

I think the approach itself is sound and useful. Adam's own observation that other implementations of the concept seem to work better confirms that.

But SyncML definitely has its flaws, both in the protocol itself and in implementations. SyncML tries to be too flexible for its own good. It allows the implementation of very dumb devices. The downside is an increased complexity on the server side.

Because of its open nature, there have been a variety of implementations with varying degrees of capabilities both in the data that is supported and in the quality of the protocol implementation itself. That makes it challenging today to support the whole range of SyncML capable peers. In that sense, SyncML is a victim of its own success.

The silver lining

I have some hopes today for CalDAV/CardDAV based synchronization. SyncEvolution 1.2 will have support for that, natively. A native implementation has the conceptual advantage that it can use meta data (resource URI + eTag) to speed up change detection, something that wouldn't be possible when going through Evolution Data Server.

CalDAV enforces that each item must have a globally unique ID, which is a considerable simplification for implementing synchronization. CardDAV unfortunately still doesn't.

Good open source implementations exist, for example Apple's Calendar server. It passes all of the automated SyncEvolution tests. I also hear good things about DAViCal; unfortunately I haven't found the time to test with it yet. What might be missing in both cases is good integration into a groupware solution, for those who need that.

I have no information about the reasons for discontinuing the service. The notification to users did not give an explanation either. My guess is that the business side of the service did not work out. Syncing is a very challenging, technically difficult problem. I suspect that many users do not understand how much hard work goes into it and thus are less willing to pay for it. Then there is the competition with free services. If users are happy with online access Google Calendar, why should they bother paying for a synchronization service? There are reasons (see below), but perhaps not enough users care.

This is really sad, for several reasons.

First, ScheduleWorld offered a technically superior SyncML implementation for calendar synchronization by supporting the full iCalendar 2.0 semantic, including UID and RECURRENCE-ID. When describing these properties in 2008, I wrote that not supporting them "looks like a rather significant gap for SyncML-based calendar synchronization compared to proprietary solutions. Hopefully SyncML server vendors will wake up and do something about it..." I don't think they have - or can anyone point to a SyncML server which supports this today, over one and a half year later? I'm not aware of any.

Second, although ScheduleWorld used the open source Funambol server code, none of the extensions or modifications were ever made available. That's perfectly compliant with the license of the code that was used (GPL instead of AGPL as today), but it implies that the ScheduleWorld source code is now lost and no-one can use or study it.

Third, I believe that storing my data on my own hardware and synchronizing it via open standards is crucial. ScheduleWorld made that possible for users who did not want to run their own SyncML server. With true syncing instead of just online access, data is available while offline - pretty obvious, but worth spelling out explicitly. When syncing is based on open standards, users can decide which service they want to use. They don't depend on that service to store their data; important, because any service might go away unexpectedly and take the data with it. With ScheduleWorld, all data was always and by design also stored locally, so no data was lost when the site shut down. The same is not true for many (if not all) of the now popular Web 2.0 services.

Replacements for ScheduleWorld

I'm not sure yet what I should recommend as replacement. Memotoo seems to be popular, but it does not support UID/RECURRENCE-ID semantic, so there are issues when synchronizing complex iCalendar 2.0 calendar data. For advanced users the SyncEvolution server mode might be an alternative, but it is in a very rough state at this point and requires a publicly accessible machine for anytime, anywhere synchronization.

A bug fix release. The main reason for releasing it is that SyncEvolution 1.0 no longer worked on recent distros (Fedora Core 13, Debian testing) because of a name clash between the Bluez D-Bus utility code and recent glib.

Details:

• compile fix for FC 13 (and possibly others): use private copy of gdbus (BMC #3556)

• sync-ui: prevent overwriting device configs by accident (BMC #3566,BMC #1194) Setting up a phone used the template name as config name and overwrote an existing configuration of another phone that was created using that same template. Now the code uses the Bluetooth device name as set on the device and checks for (less likely) collisions. It also sanitizes the name to avoid complicated config names (only relevant when also using the command line).

• syncevo-dbus-server: accept 'application/vnd.syncml+xml; charset=UTF-8' for starting an HTTP session (BMC #3554) The redundant charset specification was set by the Funambol Thunderbird client. Because of a literal comparison against 'application/vnd.syncml+xml' the messages were rejected.

• config fix: operations on non-peer configs failed (BMC #3157) When running operations on a non-peer configuration (like --restore @default addressbook), the operation fails with [ERROR] : type 'select backend' not supported

• ZYB.com: service goes away end of June 2010, template removed (BMC #3310)

• some build (BMC #2586, BMC #3557) and language updates

Source, Installation, Further information

Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources

i386, amd64 and lpia binaries of 1.0 for Debian-based distributions are available via the "stable" syncevolution.org repository. Add the following entry to your /apt/source.list, then install "syncevolution-evolution":

These binaries include the "sync-ui" GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default).

The same binaries are also available as .tar.gz and .rpm archives in http://downloads.syncevolution.org/syncevolution/evolution. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise.

After installation, follow the getting started steps.

SyncEvolution can handle synchronization with eGroupware (eGW), starting with 0.4-pre2 plus one patch, which was included in 0.4.

Make sure eGW fits the needs for correct operation with syncML, in particular php5 and Pear::Log. See here for information about requirements and settings.

The SyncEvolution setup is rather straightforward. Although the SyncML code of eGW is based on horde (see above), there's no need to restrict authentication to auth-basic.

Set syncURL to http://<host>/<egw-base>/rpc.php. Set "type = text/calendar" and "uri = ./calendar" for the calendar and "type = text/x-vcard" and "uri = ./contacts", respectively for the address book.

Warning: "uri = ./calendar" apparently is the official uri, but several users reported that with that setting events created in eGW were not copied into Evolution while "uri = calendar" worked alright. Until this is clarified, the recommendation is to use "uri = calendar".

All in all, synchronization is working well, but I'd like to mention some things I noticed not to work:

• Recurring events don't stop at a final date in eGW, even when this final date has been set in evolution.

  "A known issue of old Evolution versions." -- Patrick

• Recurrence exceptions are not respected.

  "eGW seems to rely on storing the remaining instances of an event, but does not include this information in the data it sends to SyncEvolution. On the other hand, SyncEvolution also does not yet support this either. Exceptions defined in Evolution's dialog are stored as simple properties and are sent to the server, but eGW does not seem to use that information."

• New appointments in eGW are not sync'ed to evolutions. But updates are.

  "Could not reproduce. If it still occurs, please submit a bug report."

• With SyncEvolution 0.6, Ioannis Ioannou had a problem with lost or messed up telephones. Part of the problem was in SyncEvolution (it did not correctly detect types as sent by eGW, fixed in 0.7), part of the problem was in eGW (it does not handle the vCards sent by SyncEvolution correctly). Ioannis found out that eGW has hard-coded mappings for incoming data, which might have to be adapted on a case-by-case basis for clients. He submitted a patch (thanks!) which adds a SyncEvolution mapping to eGW. Hopefully this problem will be fixed. The original version of his patch changes class.vcaladdressbook.inc.php in egroupware/addressbook/inc and was tested on 1.4.001, 1.4.002, and the trunk of 2007-09-25th.
• If communication with eGW stops with a Server Failure: server returned error code -1 error, then the reason is most likely a parser problem in eGW. Some information about this can also be found in the same discussion thread.

The Horde framework includes a SyncML component compatible with SyncEvolution and other SyncML clients. This component is built in to the core framework and does not need to be separately installed. For synchronization to work, you need to do a few things....

1. Use fresh Horde sources: Many SyncML changes are applied only to Horde's CVS HEAD branch. The stable branch does include some SyncML support, but it's not as complete as what's found in CVS HEAD. If you really don't want to use CVS HEAD sources entirely, you might be able to get away with updating just the files /horde/rpc.php and /horde/turba/lib/api.php. Try the versions from the stable CVS branch first before trying HEAD, and don't complain if they don't work correctly on top of a release installation. Much better to just use CVS HEAD altogether.
2. Configure authentication: On the Horde server, the only thing that might require configuration is authentication. By default, SyncEvolution will use md5 authentication, which is nice when you're using HTTP rather than HTTPS. Support for md5 was very recently added to Horde, and will almost certainly have changed by the time you read this. At the moment, see the comments in SyncML/Backend.php around line 404 for details. Alternatively (and more easily), run SyncML over HTTPS and tell SyncEvolution to use basic, non-md5 authentication, by adding
   clientAuthType = syncml:auth-basic serverAuthType = syncml:auth-basic

   to the ~/.sync4j/evolution/horde/spds/syncml/config.txt file (see below).

3. Configure SyncEvolution: Read the generic SyncEvolution docs, then come back here. Copy one of the prepackaged configuration directories (like etc/scheduleworld_1) to ~/.sync4j/evolution/horde as a starting point. It doesn't matter much which template directory you use. They're not very different, and there's not much to configure. You need to modify (or create) the following items. In ~/.sync4j/evolution/horde/spds/syncml/config.txt:
   syncURL = https://your.horde.site.com/horde/rpc.php

   Adjust http/https appropriately and make sure the address matches the webroot configuration on the server.

   deviceId = sc-api-nat-<identifier>

   Each computer/username combo that syncs should have a different identifier, for example, using an identifier like "home-myusername" makes sense. The username and password lines should match your Horde username and password. Add the AuthType lines from #2, if you don't want md5 authentication. Next, decide which Evolution databases you want to synchronize. I use only the addressbook, but the others should be similar. In ~/.sync4j/evolution/horde/spds/sources/addressbook_1/config.txt set

   uri = contacts

   That's it. The other components use "calendar" (Kronolith), "notes" (Mnemo), and "tasks" (Nag). If you don't want to synchronize a particular database, be sure to set "sync = none" in its config file.

4. Try it out:
   syncevolution horde

   If you get errors, check out the Horde Sync wiki for more information on debugging. The sync@lists.horde.org mailing list is also very useful. Check the archives there, to see if there's any info about your problem.

Enhancement (Note: This may be unnecessary when Turba 3 is released.)

The default configuration for Turba (Horde's address book component) is less than ideal for SyncML, because the turba_objects database table is somewhat oversimplified (for example, all components of an address are combined in a single field). This leads to vcards that are missing some fields when they're synced between Horde and Evolution. Fortunately, Turba is very clever, when it comes to reading its database, and can automagically adjust to many nondefault configurations. If you're creating a fresh Turba installation, you can improve its SyncML capabilities greatly by using this custom sources.php file. If you do this, then you'll also need to make sure the appropriate fields are available in the turba_objects table in your database. Using this postgres script, instead of the one included with Horde, will create an appropriate turba_objects from scratch. For MySQL, there is no such script available, so make the changes manually. Nothing else needs to be done with Turba, apart from using the custom sources.php and creating an appropriate turba_objects table. The Turba code will automatically recognize the new fields and use them correctly in SyncML transactions. Thanks to Karsten Fourmont for providing this solution.

Interoperability with the Funambol server was improved in SyncEvolution 0.9 by adding support for some vCard extensions (X-MANAGER/ASSISTANT/SPOUSE/ANNIVERSARY, #2418). Lost ACTION property has a work around (#2422).

To enable the full support for all data in a configuration created with SyncEvolution < 0.9, so that it exchanges items in the more suitable iCalendar 2.0 format, use:

Without the exclamation mark, format auto-negotiation would pick the less capable vCalendar 1.0 format because that is marked as preferred by the server.