Skip to content

FOSDEM 2012

I’ll give a talk about SyncEvolution and EDS in the Mobile Linux Dev Room at FOSDEM on Saturday. I hope to see you there :-)

For those who can’t come, attached are the slides.

Attachment Size
syncevolution-FOSDEM2012-lightning-talk.ppt 762.5 KB
syncevolution-FOSDEM2012-lightning-talk.pdf 419.36 KB

SyncEvolution 1.2.2 released

Maintenance release with various bug fixes.

* syncevo-dbus-server + ConnMan: fixed “online” detection (BMC #21541, BMC #24587)

SyncEvolution did not recognize any cellular connectivity as
suitable for syncing. The strict check for certain “connected
technology” is unnecessary, anything which makes the computer
“online” should be good enough. So now it just uses the ConnMan
“State” property.

Additional benefit: will continue to work with ConnMan 1.0, which
won’t have the “ConnectedTechnologies” property anymore.

The Bluetooth available check was also (incorrectly) using the
ConnMan API. Now asssume that OBEX/Bluetooth is always available.

  • automatic backups: added INFO messages and fixed dumpData/printChanges (BMC #24619)

    Point out that backups are created (user might be unaware otherwise
    and wonder about the delay), explain why (so that users know how to
    turn it off).

    Turning these backups off with dumpData=0 printChanges=0 had to be
    fixed, backups were always written previously.

  • EDS compatibility: bumped version check for EDS 3.2

    SyncEvolution is known to work with EDS 3.2. Therefore use the
    libebook/ecal/edataserver libs from 3.2 if available, without
    warnings in the –version output. Also happens with inconsistent
    distro setups where the old libs are available and would have been
    prefered by SyncEvolution 1.2.1 even though the old libs no longer
    work with EDS 3.2.

  • GTK-UI: do not accept service config without a username (BMC #23106)

    Instead of creating such a config, an error dialog is shown.

  • GTK-UI: updated translations

  • fixed various compile issues, primarily on Fedora Core 17
    (unistd.h/ssize_t, invoking syncevolution during compilation,
    missing src/dbus/qt/configure-sub.in)

    SyncEvolution is known to not compile with Bluez 4.97. A patch
    for Bluez header files is needed to make them work in C++ again,
    see http://thread.gmane.org/gmane.linux.bluez.kernel/20364

Upgrading from releases before 1.2

Old configurations can still be read. But writing, as it happens
during a sync, must migrate the configuration first. Release 1.2
automatically migrates configurations. The old configurations
will still be available (see “syncevolution –print-configs”) but must
be renamed manually to use them again under their original names with
older SyncEvolution releases.

Source, Installation, Further information

Source snapshots are in
http://downloads.syncevolution.org/syncevolution/sources

i386, lpia and amd64 binaries for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:

These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default).

The same binaries are also available as .tar.gz and .rpm archives in the download directories. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise.

After installation, follow the getting started steps.

SyncEvolution 1.2.1 released

Maintenance release with various bug fixes.

* GTK UI + config: fix “custom server” setup (BMC #13511)

When the “default” config template (= ScheduleWorld) was downgraded to
“not consumer ready” in SyncEvolution 1.1.0.99.1, setting up a custom
SyncML service in the GTK UI stopped working because the UI wouldn’t
show the “not consumer ready” config.

The problem described above is deterministic and fixed now.
Initially the problem seemed to be random. So perhaps there is
also another, related issue.

  • phone sync: delete<->delete conflict + phone calendar+todo sync (BMC #23744)

    When deleting an item on phone and locally, the next sync failed with
    ERROR messages about “object not found”. Retrying the sync then worked.

  • Nokia: prevent accidental usage of “calendar” or “todo” sources

    Nokia phones use a combined “calendar+todo” source for syncing. The
    “calendar” and “todo” sources also exist because that is where local
    databases are configured.

    In such a setup, syncing always has to use “calendar+todo”. For example,
    to refresh from the Linux desktop to the phone, use:
    –sync refresh-from-server calendar+todo

    To work with items (restore, show local content), use the underlying sources,
    as in:
    –print-items calendar

    It was possible to accidentally sync with the “calendar”. This commit
    prevents that by adding an invalid URI setting to the “calendar” and
    “todo” sources in the Nokia and Ovi templates. Existing configs are not
    touched, so beware when you already have configured your Nokia phone.

  • vCard: X- chat extensions were limited to one instance per kind

    For example, only one Jabber account could be synchronized. This
    was caused by an incomplete definition of the conversion to and from
    vCard.

  • syncevo-dbus-server + phone sync: catch SIGPIPE to avoid premature exit

    Frederik Elwert reported that running a local sync with a phone via
    Bluetooth caused the syncevo-dbus-server to shut down during a sync.
    Explicitly telling the process to ignore the SIGPIPE signal solved that
    problem.

  • syncevo-http-server: support chained SSL certificates

    So far, the file pointed to by –certificate-file had to
    contain the server certificated (signed by a CA known to the client)
    and (optionally) a client certificate. Now the file may also contain
    additional intermediate certificates which will be sent to the client
    (chained certificates).

  • documentation: added glossary and command line conventions sections,
    improved listing of properties, embedd property definitions in man page,
    README and README.html

  • EDS compatibility: fixed inconsistency in libecal check

    The check for the _r variants in libical still used an older max
    version. This might have prevented using them (if not found) or
    could have led to a mixture of old and new libecal in the same
    process (probably crashed).

  • glib: avoid including glib/*.h headers directly

    Recent glib deprecates the direct inclusion of some of its headers,
    in favor of including glib.h. Doing that here whenever possible, so
    perhaps it now compiles on Fedora 17 (untested).

Upgrading from releases before 1.2

Old configurations can still be read. But writing, as it happens
during a sync, must migrate the configuration first. Release 1.2
automatically migrates configurations. The old configurations
will still be available (see “syncevolution –print-configs”) but must
be renamed manually to use them again under their original names with
older SyncEvolution releases.

Source, Installation, Further information

Source snapshots are in
http://downloads.syncevolution.org/syncevolution/sources

i386, lpia and amd64 binaries for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:

These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default).

The same binaries are also available as .tar.gz and .rpm archives in the download directories. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise.

After installation, follow the getting started steps.

State of the union, version 1.2

With SyncEvolution 1.2 released and work on 1.3 under way it is a good time to take a small break and reflect on the state of the SyncEvolution project.

Features

This is meant to give an idea of what can be done with SyncEvolution already. For a more complete list of features, see the documentation about backends, frontends and engine.

This section introduces the key ideas, some of the supported protocols + backends and how they work in combination with specific peers.

Data Synchronization

The main purpose of SyncEvolution is the synchronization of Personal Information Management (PIM) data (contacts, events, tasks, notes). This intentionally does not include email and messages in general because those are sufficiently different to require different solutions. SyncEvolution provides real synchronization of PIM data:

  • Items (= contact, event, …) may have different properties and representations on both sides of a sync (heterogeneous environment).
  • Items can be modified at any time, even while offline (more than just online access to a server).
  • Also works between devices, without involving a server.
  • A database (= a set of items) can be synchronized to one or more peers (other devices, servers). For example, the same address book on a desktop can be synchronized with a mobile phone and a laptop.
  • SyncEvolution supports arbitrary number of databases. However, many peers (in particular SyncML servers) are more limited and only provide access to one addressbook, calendar, etc.

For each peer, SyncEvolution needs to remember the previous state so that the next sync can be limited to only those items which have changed. The underlying assumption is that there is no cycle in the connections between multiple sync peers. For example, “phone <-> laptop, laptop <-> server, phone <-> server” is bad because a new item created on the phone will go to the laptop, from there to the server and then come back as new item to the phone, resulting in a duplicate which then repeats the process ad infinitum. Each SyncEvolution instance only knows the peers it is set up to talk with and thus cannot detect the problem.

To break this cycle reliably, each item must have a unique ID that is assigned to it once when the item gets created. For ad-hoc synchronization to work without constraints on the topology, this ID must be supported by all involved peers. Unfortunately, calendar and contact data either doesn’t have such an ID or it is not supported. Without such an ID, each request to add a new item must be checked against all existing items based on some key properties to find out whether it is a duplicate. This is slow and error-prone (”Is ‘Doe, John” the same person as ‘John Doe’, or is one the father and the other a child or namesake?”).

TODOad-hoc synchronization between SyncEvolution instances”:

  • Teach Evolution Data Server to create and preserve a unique ID for contacts (the vCard 3.0 UID gets overwritten at the moment).
  • Improve the SyncEvolution<->SyncEvolution sync such that
    • it uses the unique ID to speed up sync between peers which sync against each other for the first time and
    • checks the ID to find duplicates in following syncs.

TODOad-hoc synchronization between arbitrary peers”:

  • Do the duplicate detection based on item content for each add request.

Testing

SyncEvolution has an extensive test suite which is run regularly (used to be nightly, currently triggered manually). It contains unit tests for various aspects of SyncEvolution and does real interoperability testing with different SyncML, CalDAV and CardDAV servers. See the SyncEvolution 1.2 test report for an example. More CalDAV/CardDAV servers were added in the 1.3 branch.

Keeping this testing going and analyzing/reporting problems is an on-going activity. It is needed to avoid regressions and achieve higher quality in the releases, which are built as part of these test runs. There are additional ideas.

TODO “improve nightly testing”

SyncML + Engine

The Synthesis engine which provides SyncML is one of the best in the industry. It has extensive support for data modeling/conversion and supports suspending a session and resuming it later.

The same engine is also used to synchronize between two backends internally. This is how synchronization was added for CalDAV/CardDAV, protocols which themselves only provide online access. This works reasonably well, but there are also quite a few limitations.

TODOlibvxx = refactoring of the Synthesis code base

  • make data conversion available outside of a sync session
  • decouple sync engine from SyncML and SyncML message encoding/decoding
  • support items which are a set of items: important for CalDAV + ActiveSync, because those combine all VEVENTs with the same UID in one item; certain transformations cannot be done in the engine at the moment because they depend on access to all related items at once and that is not how they are handled at the moment
  • more flexible session handling: instead sending changes in one direction, then back and then stopping, allow the session to continue until both sides are in sync; required for CalDAV where storing an event might lead to further changes that have to be sent back

TODOpush sync“:

  • react to local or remote changes immediately (instead of polling at long intervals) and/or
  • make sync sessions without changes more efficient (in particular when polling)

TODObetter credential handling

  • When creating multiple configurations which need the same credentials (Google CalDAV and SyncML, for example), the username/password needs to be set separately. Replace with a mechanism where both configs only contain a pointer to shared credentials.
  • Also support an external system component which does the authentication without ever returning username/password to SyncEvolution. Depends on having such a component.

This could be used to use a CalDAV/CardDAV server as backend for a SyncML server.

TODOuse CalDAV/CardDAV inside SyncML server“:

  • In the backend, check databaseuser/password before falling back to the context’s username/password.
  • Support shared credentials, to avoid having to configure them in each source.

At the moment there are situations where SyncEvolution cannot determine what the right resolution for a failed synchronization is. It has a built-in backup mechanism and can ask the user for assistance, but ultimately it would be better to not bother the user. This can be achieved in some cases by simplifying the problem:

  • The peer (most likely a service on the Internet) must be able to store all data.
  • There is a separate local database for each database on the peer.

In that case it is acceptable to wipe out the local data and restart with the data stored on the peer.

TODOautomatic error recovery

  • Implement the necessary policy in SyncEvolution (”peer wins”).
  • Check whether this interferes with libfolks (local IDs will change, data added by libfolks to a contact might get lost).

CalDAV/CardDAV

Works reasonable well and passes automated testing against a variety of servers (Apple Calendar Server, DAViCal, Google Calendar, Yahoo). But there are some know limitations, like meeting invitations being sent by Evolution and the CalDAV server.

TODObetter support for meeting invitations

  • Find a way to suppress sending of meeting invitations on the CalDAV server when Evolution already sent one, or
  • suppress sending of meeting invitations in Evolution for calendars which are mirrored in a CalDAV server.
  • When storing a meeting and/or the server does not return an ETag, retrieve the possibly modified item from the server and store the modified item locally. Right now the more possibly automatically modified data on the server gets ignored. Depends on the engine improvements mentioned above.

TODOhandle concurrent changes

  • Use ETags to avoid modifying more recent data on the server.

TODOCalDAV attachments

  • At the moment, attachments are not supported at all and even may get lost. Need to preserve them and perhaps even support the more efficient “managed attachments” that are currently being discussed by the CalConnect consortium.

TODOWebDAV: use sync extension

  • Each sync session must list the entire collection (= retrieve path names and ETag) to determine new and modified items. This can be done more efficiently by using the sync extension defined and implemented in the Apple Calendar Server.

ActiveSync

This was not included in SyncEvolution 1.2 although it was already quite usable. Work on it is still going on. The main issue with ActiveSync is the limited data model specified as part of the protocol.

Contacts are only allowed to have a fixed number of certain phone numbers and addresses, which is a limitation that neither Evolution nor Google Contacts have. There is no good way to handle these limitations except educating the user about them, or enforcing the same constraints locally by modifying the app which creates and modifies contacts.

The ActiveSync calendar format does not support detached recurrences properly (”you are invited to a specific instance of a meeting series”), although Exchange internally does.

TODO “finish initial ActiveSync support”

  • detached recurrences without parent:
    • investigate receiving multiple detached recurrences in multiple items with the same UID in each (the Exchange workaround for the ActiveSync calendar format limitation): might break SyncEvolution
    • implement the “stand-alone detached recurrence” support (either do it like Exchange does or better, create a fake parent event)
  • test concurrent item changes while a sync runs (supposed to work, but without a test case it is hard to be sure)

[updated] TODO “ActiveSync performance improvement”

  • Writing changes to the server is done one change at a time. Could be improved considerably by batching changes. Depends on core engine improvements.

TODOActiveSync push sync

  • enhance activesyncd and SyncEvolution (see engine above) to react to server-side changes with minimal delay

TODOActiveSync calendar attachments

  • At the moment, attachments are not supported at all and even may get lost. Need to preserve them. Not sure how to do it efficiently.

Google Calendar

Overall Google Calendar syncing works well with CalDAV (I’m using it myself) as long as one does all meeting scheduling in the Google web interface. But there are some known issues, most of them on the server side:

Google Contacts

Works via SyncML. Google’s support for SyncML is very incomplete (many properties not supported, for example birthday).

TODO “better Google Contacts support”

Apple Calendar Server, DAViCal

Done via CalDAV/CardDAV. No know issues with these peers.

Yahoo Calendar + Contacts

Works well, when it works; unfortunately the number of requests per 24 hour period is so limited that the tests cannot complete without running into a 503 “Service Unavailable” error.

TODOYahoo token authentication

  • Currently SyncEvolutions uses normal HTTP authentication. Yahoo also supports another, token based authentication mechanism for approved apps. Get SyncEvolution approved and implement that other authentication mechanism. Hopefully that’ll avoid the 503 error.

Phones

SyncEvolution can synchronize against phones if those phones support SyncML via Bluetooth. This is a common feature among older feature phones but most (all?) Android phones and iPhones don’t support it. Testing of this feature is limited and there is no list of phones which are guaranteed to work.

[updated] Synchronization with Android/iOS is possible by installing a third-party SyncML client (like the ones from Synthesis) and configuring the phones to use a SyncEvolution HTTP SyncML server.

TODOcontact sync via PBAP

  • Write a backend using PBAP, the only (?) protocol supported by Android and iOS for address book access. Not very good for real syncing, but at least one-way sync should be possible.

GTK UI

The GTK “sync UI” is the main user interface for SyncEvolution in MeeGo and Linux in general. It is included in the source and binary distribution archives. It supports configuring syncing against SyncML services (extensible via configuration templates) and phones which support SyncML. Configuring the latter is integrated into the GNOME Bluetooth applet, which invokes the sync UI. The UI also has “emergency recovery” support which allows the user to restore from the automatic backup and/or choose between different recovery operations after a failed sync:

  • continue with local data
  • continue with remote data
  • try to merge both (”slow sync”)

TODOport to GTK3

  • The sync UI is currently using GTK2. It needs to be ported to GTK3. The goal is to put the GTK2 version into maintenance and continue with the GTK3 version.

TODOconfigure local sync, choose databases

  • CalDAV/CardDAV and ActiveSync cannot be configured yet with the UI. In this context it becomes important to let the user choose local and remote databases. Would be useful to have, although there is a certain overlap with the Evolution integration.

Evolution

This happens to be the main backend for storing data locally. Other backends could be supported just as well (there’s nothing technical which favors Evolution) if there were developers motivated enough to implement and test them. KDE/Akonadi has come a long way, but seems less active (see KDE/Akonadi and Community below).

TODO “Evolution backend improvements”

TODOintegrate SyncEvolution into Evolution

  • Evolution typically supports offline read access with some of its backends. It does not support write access. SyncEvolution addresses that, but needs to be configured and invoked separately. It would be nice to have seamless and transparent syncing from inside Evolution. ActiveSync is getting integrated like that at the moment -> extend that to CalDAV/CardDAV?

File backend

There is a generic file backend with a 1:1 mapping between a single item and a file in a directory. The format of the local data can be configured. vCalendar 1.0, iCalendar 2.0, vCard 2.1/3.0 and plain text notes are supported already.

TODOiCalendar 2.0 .ics file

  • Some people synchronize an .ics file by pointing the Evolution backend to it (no longer works after EDS removed support for the file:// URI) or manipulating the ~/.evolution/calendar data (a hack which only works when being very careful). Write a SyncEvolution backend which reads/writes an iCalendar 2.0 file using libical.

KDE/Akonadi

The Akonadi backend is available in the source code. Support for KWallet instead of GNOME keyring is also there. A KDE GUI is in development.

TODOtest and release binaries

  • nightly builds needs to be reconfigured to enable building the KDE support
  • nightly testing needs to include KDE backends, both for unit testing the backends and real combinations with peers; it depends on the KDE developers to do something if issues are found in those tests (if any are found)

Manipulating PIM data

SyncEvolution also is a very capable tool for manipulating databases via the command line. It can list, import/export and delete items. Converting between formats (for example, mirror Evolution contacts as files in vCard 2.1 format) is possible by setting up synchronization with a file backend as peer.

TODO “improve item manipulation”

Community

Lots of users. The feedback from users is often very helpful for improving the software.

Not so many developers, though, except those paid to work on SyncEvolution. Ove Kaaven (port+UI for Maemo/N900 and MeeGo Harmattan/N9/N950) and Frederik Elwert (Genesis UI) are the notable exceptions. Sascha Peilicke/Dinesh/Rohan Garg have done some work for KDE/Akonadi, but are not very active in the SyncEvolution project itself.

I can only speculate about the reasons for the lack of external contributors:

  • The number of people who care about PIM sync and storage is small to start with. It’s not one of those sexy areas that gets a lot of attention, although it is arguably very important.
  • OpenSync still seems to be considered “the” open source sync solution, despite not having a stable release available or even in sight anytime soon.
  • Perhaps developers also get the impression that problems will be solved anyway, without having to get involved. That is true for some aspects, but definitely not for all.
  • SyncEvolution development has a certain learning curve (although Ove and Franz Knipp managed to write their backends with very little assistance) and often happens at a rapid pace, which makes it hard for people to contribute small improvements.
  • SyncEvolution was focused on Evolution and SyncML initially, which might make it seem too limited in scope for some use cases. But the scope has already increased and will increase further, so now would be a good time to check it out again.

Many of the TODOs above will not get worked on unless some external developer picks up the mantle and contributes patches. I’d be more than happy to help someone get started. If there is interest, I can also tag bugs in Bugzilla as “easy fixes”. Right now I am not doing that because it would create additional work which would not be justified if there is no-one interested – I know, chicken and egg…

SyncEvolution 1.2 released

The major new feature of the 1.2 release is support for non-SyncML
protocols in general and CalDAV/CardDAV in particular. ActiveSync
support is in development and will be in 1.3. These protocols are
implemented as backends which are combined with other backends by
SyncEvolution in a so called “local sync”. The GTK sync-ui does not
yet support configuring non-SyncML protocols. See the README.rst and
man page
for more information on how to use the new feature via the
command line.

Properties not supported by SyncML servers can now be preserved
locally in two-way synchronization (BMC #15030). This depends on
information about what properties a SyncML server supports (”CtCap”),
which is typically not provided by servers. SyncEvolution contains a
copy of that information for Google Contacts (BMC #15029).

Akonadi backend and KWallet support were merged. They are not included
yet in syncevolution.org binaries. To use them compile from source.

The configuration format was updated to solve a conceptual problem
inherited with the legacy property names: the “type” property had
multiple, sometimes conflicting roles. For example, setting the
preferred data format for sync with one peer might have changed the
backend selection for some other peer (BMC #1023). Now
“backend/databaseFormat/syncFormat/forceSyncFormat” replace
“type”. “type” is still accepted by the command line as alias.

Upgrading from releases before 1.2

Old configurations can still be read. But writing, as it happens
during a sync, must migrate the configuration first. Release 1.2
automatically migrates configurations. The old configurations
will still be available (see “syncevolution –print-configs”) but must
be renamed manually to use them again under their original names with
older SyncEvolution releases.

Other changes

  • Using the –sync-property and –source-property command line options is
    optional, just specifying the property assignment is enough.

  • syncevo-http-server was enhanced considerably. See http://syncevolution.org/wiki/http-server-howto

  • support NetworkManager API >= 0.9 (BMC #19470)

  • syncevolution.org binaries: now compatible with Debian Testing/libnotify.so.4 (BMC #22668)

    libnotify is not linked directly into syncevo-dbus-server in the
    syncevolution.org binaries. Instead libnotify.so.1 till .so.4
    (current Debian Testing) are opened opened dynamically and the
    necessary functions are looked up via dlsym(). Not finding the
    libraries or the functions silently disables this notification
    mechanism.

  • Sync mode is recorded when running in SyncML server mode (BMC #2786).

  • syncevo-dbus-server automatically stops when some of its libraries
    are updated and restarts if auto-syncing is on (BMC #14955).

  • Added code for Buteo, mKCal and QtContacts in MeeGo.

    Buteo and mKCal were removed again from MeeGo, so the code
    is obsolete. The QtContacts backend may be still be useful
    to access items via that API, but for syncing on MeeGo
    the normal EDS backend is used since MeeGo reverted back
    to EDS as PIM storage.

  • “databasePassword” source property: lookup failure in keyring (BMC #22937)

    The databasePassword also wasn’t looked up at all when doing item operations
    via the command line.

    When configuring sources for an HTTP server, the config name typically
    is just the context (@foo). When using the config in the HTTP server,
    the config name is the peer inside that context (client@foo). Because
    the GNOME keyring lookup keys for the “databasePassword” (more
    specifically, the object name) contained the full config name which
    was different in both cases, looking up the saved password failed.

    The solution is to normalize the config name (to accomodate for
    different ways of spelling it) and use only the context, with @ as
    before. This will break existing setups where the object name in the
    keyring (incorrectly) includes the full config name. In that case just
    configure the source again to set the password anew.

  • Evolution Calendar: fixed detached recurrence support (BMC #22940)

    When manipulating a meeting series with more than one detached
    recurrence certain sequences of operations could incorrectly fail
    with “UID already exists”.

  • iCalendar 2.0: must set VALUE in EXDATE (part of BMC #22940)

    EXDATE has a VALUE parameter, which wasn’t defined in the XML
    profile. Didn’t seem to matter at all in practice, but wasn’t
    standard-compliant.

  • GTK sync-ui: wrap sync service descriptions (BMC #7199)

    Descriptions of different sync services are not fully visible unless
    word-wrapping gets enabled.

  • CalDAV/CardDAV + local storage: avoid empty properties

    The main motivation for this change is that a recent Apple Calendar
    server rejects vCards with empty BDAY property. Another reason is that
    keeping the data as small as possible is desirable by itself.

    Sending an empty property serves as a hint for the peer that the
    property is supported. This is not necessary when storing an item in a
    backend. Therefore this commit disables empty properties for all
    backends which do not themselves set the m_backendRule Synthesis info
    value.

  • Google Contacts: ensure that first/middle/name are set when storing in EDS (BMC #20864)

    Evolution and the MeeGo UX assume that first/middle/last name are set.
    That is not the case when a contact is created in the Google Contacts
    web interface. Such contacts are sent by Google without the N
    property.

    SyncEvolution now tries to recreate the name components from the FN
    string, by splitting at word boundaries and assuming ”
    ” or “, ” format. Obviously this
    heuristic fails for some locales.

  • Evolution Calendar: fixed error handling for broken TZIDs

  • Sony Ericsson: use ISO-8859-1 for all devices (BMC #14414)

    Passing invalid UTF-8 strings into libecal caused glib to
    abort syncevo-dbus-server.

  • auto sync: show all failed syncs except for temporary network errors (BMC #21888)

    Notifications were meant to be shown for all errors except temporary
    ones. This has never been implemented correctly since the feature was
    introduced: instead of hiding known temporary errors, all errors except
    500 (fatal error) were suppressed.

  • vCard: inline local photo data (BMC #19661)

    Some platforms (Maemo, MeeGo) store photos in separate files. Now SyncEvolution
    efficiently includes that photo data in the generated vCard right before sending
    it to a peer; previously it sent a useless local file:// URI. The Maemo port
    has a less efficient workaround for that which now should be obsolete.

  • syncevo-dbus-server: online status wrong without Network Manager or ConnMan (BMC #21543)

    When neither Network Manager nor ConnMan are running, network presence was “not
    online”. This prevented running automatic syncs.

For developers:

  • modified backend API

    • ClientTestConfig modernized
    • InsertItemResult::m_merged turned from boolean to enum
  • testing and compilation changes; for example, the minimum version of
    libsynthesis is now checked at configure time instead of failing at
    runtime due to missing features in the Synthesis engine

SyncEvolution 1.1.99.7 -> 1.2, 13.10.2011

Some more bug fixes and testing improvements.

  • fixed potential invalid memory access in add<->add conflict handling
  • fixed memory leak in workaround for EDS bug
  • CalDAV/CardDAV: handle ETags without quotation marks (eGroupware)
  • updated README: warning about sync direction moved to –sync option

Source, Installation, Further information

Source snapshots are in
http://downloads.syncevolution.org/syncevolution/sources

i386, lpia and amd64 binaries for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:

These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default).

The same binaries are also available as .tar.gz and .rpm archives in the download directories. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise.

After installation, follow the getting started steps.

SyncEvolution + non-recursive Automake

Krzesimir Nowak wrote about his work on converting SyncEvolution from an autotools project with recursive make to non-recursive make. Definitely worth a read for anyone interested in autotools. The current SyncEvolution master branch (post 1.2) uses that new build system.

His conclusion is that the new system is not necessarily easier to understand than the one before (autotools with some preprocessing shell scripts). Partly that’s because SyncEvolution tries to achieve certain things not supported well by autotools (automatically generated version number, avoid listing all files explicitly, backends which can add to the global configure script), partly it is because non-recursive Automake introduces additional constraints (like having to avoid clobbering variables and rules). Helper scripts are still needed, the only difference is how they get called.

Either way, what sold me on the idea of a non-recursive make is that on a machine with many cores, like the SyncEvolution nightly test server, compilation is considerably faster because parallel make can spawn more jobs in parallel. Recursive make often had to wait for the completion of compilation in a sub-directory. If memory serves me right, it was more than twice as fast.

SyncEvolution 1.1.99.7 released

Mostly bug fixes again. Some are a bit more intrusive, thus another
pre-release.

Changes

  • syncevolution.org binaries: now compatible with Debian Testing/libnotify.so.4 (BMC #22668)

    libnotify is not linked directly into syncevo-dbus-server in the
    syncevolution.org binaries. Instead libnotify.so.1 till .so.4
    (current Debian Testing) are opened opened dynamically and the
    necessary functions are looked up via dlsym(). Not finding the
    libraries or the functions silently disables this notification
    mechanism.

  • calendar sync: better handling for add<->add conflicts (partly fixes BMC #22783)

    When both sides of a sync have added the same event, the sync must
    determine which one is more recent instead of blindly overwriting
    always the same side. Such conflicts are typically rare except for
    enterprise scenarios where meeting invitiations are processed
    automatically by a groupware (Exchange, Google Calendar/Mail, …)
    and then the attendee status is updated on one side.

    SyncEvolution now does the necessary age comparison and preserves the more
    recent data for most properties. In some properties the data from both
    sides is preserved by concatenating the text (description, location, …).
    It remains to be seen whether that is really desirable. Also, sync statistics
    are slightly off: the incoming item is counted as “added” even though it
    gets turned into an update.

  • item operations: authentication problem for WebDAV when using keyring (BMC #21311)

    The password still wasn’t looked up in the keyring when using
    –import/export/delete-items.

  • “databasePassword” source property: lookup failure in keyring (BMC #22937)

    The databasePassword also wasn’t looked up at all when doing item operations
    via the command line.

    When configuring sources for an HTTP server, the config name typically
    is just the context (@foo). When using the config in the HTTP server,
    the config name is the peer inside that context (client@foo). Because
    the GNOME keyring lookup keys for the “databasePassword” (more
    specifically, the object name) contained the full config name which
    was different in both cases, looking up the saved password failed.

    The solution is to normalize the config name (to accomodate for
    different ways of spelling it) and use only the context, with @ as
    before. This will break existing setups where the object name in the
    keyring (incorrectly) includes the full config name. In that case just
    configure the source again to set the password anew.

  • Evolution Calendar: fixed detached recurrence support (BMC #22940)

    When manipulating a meeting series with more than one detached
    recurrence certain sequences of operations could incorrectly fail
    with “UID already exists”.

  • iCalendar 2.0: must set VALUE in EXDATE (part of BMC #22940)

    EXDATE has a VALUE parameter, which wasn’t defined in the XML
    profile. Didn’t seem to matter at all in practice, but wasn’t
    standard-compliant.

  • GTK sync-ui: wrap sync service descriptions (BMC #7199)

    Descriptions of different sync services are not fully visible unless
    word-wrapping gets enabled.

  • source configs: don’t check “backend” unless it is needed

    When using a config which has sources with a backend type set which is
    not currently available, an error was thrown even if those sources
    weren’t even part of the current operation (for example, syncing
    another source which is currently supported).

  • config migration: avoid name conflicts and auto syncing of old configs (BMC #22691)

    When (auto-)migrating a config, it was possible that a name for the
    peer, say foo.old, was chosen for the renamed config although there
    was already such a config, for example foo.old in ~/.sync4j. Besides
    being confusing for users, this also led to a bug in the code where it
    copied from the older config with the foo.old name.

    The main problem fixed is the disabling of auto syncing
    in the old config. Otherwise it was still used by syncevo-dbus-server
    for syncing, which triggered another auto-migration, ad infinitum…

  • auto syncing: must check whether enabled when looking at unknown URLs (part of BMC #22691)

    “syncURL = insert your URL here” with “autoSync = 0″ did lead to auto
    sync attempts although it wasn’t enabled. A check for “auto syncing
    enabled” was missing for the “unknown transport” case.

  • CalDAV/CardDAV + local storage: avoid empty properties

    The main motivation for this change is that a recent Apple Calendar
    server rejects vCards with empty BDAY property. Another reason is that
    keeping the data as small as possible is desirable by itself.

    Sending an empty property serves as a hint for the peer that the
    property is supported. This is not necessary when storing an item in a
    backend. Therefore this commit disables empty properties for all
    backends which do not themselves set the m_backendRule Synthesis info
    value.

  • Apple CardDAV: apply PHOTO import/export scripts by default

    A recent Apple Calendar server (correctly) rejects the invalid
    PHOTO;TYPE=unknown: property in a vCard. This internal representation
    must be cleared before serializing the field list.

  • for developers: modified backend API

    • ClientTestConfig modernized
    • InsertItemResult::m_merged turned from boolean to enum
  • testing and compilation changes; for example, the minimum version of
    libsynthesis is now checked at configure time instead of failing at
    runtime due to missing features in the Synthesis engine

Source, Installation, Further information

Source snapshots are in
http://downloads.syncevolution.org/syncevolution/sources

i386 and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:

Binaries for lpia will be made available again in the next release.

These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default).

The same binaries are also available as .tar.gz and .rpm archives in http://downloads.syncevolution.org/syncevolution/evolution. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise.

After installation, follow the getting started steps.

SyncEvolution 1.1.99.6 released

Mostly bug fixes, some improvements in testing and packaging. This
release was tested successfully with DAViCal 0.9.9.4.

Changes

  • CalDAV: fixed incorrect change tracking causing “event not found” (BMC #22329)

  • CalDAV: handle delete<->delete conflict during local sync (BMC #22327)

    If the same event was deleted both locally and in the CalDAV server, syncing
    failed with “event not found”.

  • Google Contacts: ensure that first/middle/name are set when storing in EDS (BMC #20864)

    Evolution and the MeeGo UX assume that first/middle/last name are set.
    That is not the case when a contact is created in the Google Contacts
    web interface. Such contacts are sent by Google without the N
    property.

    SyncEvolution now tries to recreate the name components from the FN
    string, by splitting at word boundaries and assuming ”
    ” or “, ” format. Obviously this
    heuristic fails for some locales.

  • CalDAV: continue despite Google Calendar access problems (see BMC #19484)

    An attempt to work around “403 You don’t have access to change that
    event” errors, perhaps caused by
    http://code.google.com/p/google-caldav-issues/issues/detail?id=38
    The problem is now recorded instead of aborting the sync. The sync
    then ends in a 22001 = “partial failure” error and the operation
    will be retried in the next sync.

  • CalDAV: transform UTC RECURRENCE-ID for Evolution (BMC #22594)

    Evolution showed a meeting twice on the day of a modified recurrence,
    if the meeting series was originally created and modified in Exchange,
    then imported into Google Calendar.

  • CalDAV syncevolution.org binaries now works when libneon.so.27
    or libneon-gnutls.so.27 (Debian) are installed. Previously
    libneon.so.27 was required, which is no longer available in
    Debian Testing.

  • syncevo-dbus-server/gdbus: fixed segfault when asked for properties
    when none are available (BMC #22152)

  • Evolution Calendar: fixed error handling for broken TZIDs

  • Sony Ericsson: use ISO-8859-1 for all devices (BMC #14414)

    Passing invalid UTF-8 strings into libecal caused glib to
    abort syncevo-dbus-server.

  • item operations: authentication problem for WebDAV when using keyring (BMC #21311)

    The password wasn’t looked up in the keyring when using –print-items/import/export/…

  • WebDAV: fixed item operations without configuration (BMC #22164)

    Previously failed with “[ERROR] : virtual read-only configuration node, cannot write
    property webDAVCredentialsOkay = 1″.

  • auto sync: show all failed syncs except for temporary network errors (BMC #21888)

    Notifications were meant to be shown for all errors except temporary
    ones. This has never been implemented correctly since the feature was
    introduced: instead of hiding known temporary errors, all errors except
    500 (fatal error) were suppressed.

  • vCard: inline local photo data (BMC #19661)

    Some platforms (Maemo, MeeGo) store photos in separate files. Now SyncEvolution
    efficiently includes that photo data in the generated vCard right before sending
    it to a peer; previously it sent a useless local file:// URI. The Maemo port
    has a less efficient workaround for that which now should be obsolete.

  • syncevo-dbus-server: online status wrong without Network Manager or ConnMan (BMC #21543)

    When neither Network Manager nor ConnMan are running, network presence was “not
    online”. This prevented running automatic syncs.

  • fixed compile issues with Debian Testing/gcc 4.6.1

Known issues, might still be resolved for the final 1.2

  • syncevolution.org binaries: libnotify1 -> libnotify4 incompatibility (BMC #22668)

    Newer distros no longer have the libnotify.so.1 that syncevolution.org
    binaries depend on. As a workaround it is possible to install the libnotify1
    package from older distro releases.

  • CalDAV: add<->add conflicts (BMC #22669)

    Suppose the same meeting invitation for event UID=FOO is processed in
    both Evolution and Google Calendar. This always happens when the meeting
    invitation emails is sent to Google Mail, then later viewed in Evolution.
    On the Evolution side, the invitation is accepted. In Google Calendar this is
    still open.

    When syncing in that state the sync engine does not recognize that
    both sides have added the same meeting and the “meeting accepted”
    information eventually gets lost.

    As a workaround, always synchronize the calendar before processing
    meeting invitation emails.

Source, Installation, Further information

Source snapshots are in
http://downloads.syncevolution.org/syncevolution/sources

i386 and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:

Binaries for lpia will be made available again in the next release.

These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default).

The same binaries are also available as .tar.gz and .rpm archives in http://downloads.syncevolution.org/syncevolution/evolution. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise.

After installation, follow the getting started steps.

SyncEvolution 1.1.99.5 “beyond SyncML” released

Release 1.1.99.5 is the first release candidate for 1.2. It has gone
through a long stabilization period and thus is suitable for normal users.

The major new feature of the 1.2 release is support for non-SyncML
protocols in general and CalDAV/CardDAV in particular. ActiveSync
support is in development.

Changes 1.1.1 -> 1.1.99.5

The major new feature of the 1.2 release is support for non-SyncML
protocols in general and CalDAV/CardDAV in particular. ActiveSync
support is in development. These protocols are implemented as backends
which are combined with other backends by SyncEvolution in a so called
“local sync”. The GTK sync-ui does not yet support configuring
non-SyncML protocols. See the README and man page for more information
on how to use the new feature via the command line.

Properties not supported by SyncML servers can now be preserved
locally in two-way synchronization (BMC #15030). This depends on
information about what properties a SyncML server supports (”CtCap”),
which is typically not provided by servers. SyncEvolution contains a
copy of that information for Google Contacts (BMC #15029).

Akonadi backend and KWallet support were merged. They are not included
yet in syncevolution.org binaries. To use them compile from source.

The configuration format was updated to solve a conceptual problem
inherited with the legacy property names: the “type” property had
multiple, sometimes conflicting roles. For example, setting the
preferred data format for sync with one peer might have changed the
backend selection for some other peer (BMC #1023). Now
“backend/databaseFormat/syncFormat/forceSyncFormat” replace
“type”. “type” is still accepted by the command line as alias.

Old configurations can still be read. But writing, as it happens
during a sync, must migrate the configuration first. In contrast to
earlier, more experimental releases in the 1.2 series, 1.1.99.5 and
later automatically migrate configurations. The old configurations
will still be available (see “syncevolution –print-configs”) but must
be renamed manually to use them again under their original names with
older SyncEvolution releases.

Other changes:

  • a problem with enabling release mode required replacing 1.1.99.5
    with a fixed 1.1.99.5a release
  • syncevo-http-server was enhanced considerably. See the HTTP server HOWTO
  • support NetworkManager API >= 0.9 (BMC #19470)
  • Sync mode is recorded when running in SyncML server mode (BMC #2786).
  • syncevo-dbus-server automatically stops when some of its libraries
    are updated and restarts if auto-syncing is on (BMC #14955).
  • Using the –sync-property and –source-property command line options is
    optional, just specifying the property assignment is enough.
  • Added support for Buteo, mKCal and QtContacts in MeeGo.
    Buteo and mKCal were removed again from MeeGo, so the code
    is obsolete. The QtContacts backend may be still be useful
    to access items via that API, but for syncing on MeeGo
    the normal EDS backend is used since MeeGo reverted back
    to EDS as PIM storage.
  • code cleanup and various minor fixes/improvements, see ChangeLog

Source, Installation, Further information

Source snapshots are in
http://downloads.syncevolution.org/syncevolution/sources

i386, amd64 and lpia binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:

These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default).

The same binaries are also available as .tar.gz and .rpm archives in http://downloads.syncevolution.org/syncevolution/evolution. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise.

After installation, follow the getting started steps.

SyncEvolution for Debian: new maintainer needed

David Bremner, the Debian packager of SyncEvolution, announced that he is looking for a new maintainer to take over that package. If you care about SyncEvolution in Debian and Ubuntu, then please consider taking over.

On this occasion let me thank David Bremner for getting SyncEvolution into Debian and for maintaining it there.