Skip to content

SyncEvolution 0.9 beta 1: Full-time Project, using Synthesis SyncML Engine

When I started this blog, I wanted to keep it limited to my private open source work and not write about the work that I do as part of my regular software engineering job at Intel. That distinction is mute now because as of January this year, my main job at Intel is about SyncEvolution and data synchronization in Moblin. That means I now have the chance to work on it not just in the evening and weekend, but also during the day. No wait, I mean, I can work on it during regular work hours and have my normal life back in the rest of the time…

This change is an opportunity for SyncEvolution to go beyond its traditional role of a simple SyncML client for Evolution. It’s too early to announce specific plans, but the intention is to include a GUI and also work on direct device-to-device and device-to-desktop synchronization. This will all be open source, so if you want to help, then let’s talk (see below).

Moving to Synthesis SyncML Engine

The other big change in direction is that SyncEvolution 0.9 will be based on the Synthesis SyncML Engine, which is getting re-released under the LGPL by Synthesis. The Funambol C++ library, the basis for SyncEvolution in previous releases, has served SyncEvolution well. SyncEvolution wouldn’t exist if Funambol hadn’t released their source code. I hope that my contributions to Funambol (port to Linux+autotools, interoperability testing, CPPUnit testing framework, to name just some of them) will continue to be useful even though now it is time to move on and I’m not going to contribute directly anymore.

I look forward to similarly productive cooperation with Synthesis. So far, I have added the autotools build scripts to it, made sure that it compiles cleanly with gcc on 32 and 64 bit, and fixed a few issues that came up due to the different way of using the source in SyncEvolution.

Why change a running system? It’s because the Synthesis code base is arguably a better match for SyncEvolution’s goals. The core engine is written in C++ and implements both the client *and* the server side of the synchronization. The Funambol server is implemented as a Java Enterprise application and depends on an application server like Tomcat. The bundle made available by Funambol of course runs on normal desktops, but this is not a replacement for a normal desktop application. Refactoring the server so that its SyncML engine can be used without an application server would be a lot of work and likely lead to patches that cannot be accepted upstream because they change the core architecture.

There are several other advantages that come from switching to the Synthesis engine:

  • suspend/resume support (not used yet in 0.9 beta 1)
  • description of SyncEvolution capabilities are generated automatically
  • WBXML, the more efficient encoding of SyncML messages, is now supported (but disabled by default in 0.9 beta 1 for further testing)
  • SyncEvolution can and will be licensed as LGPL, which is a better choice for a project that is meant to become the infrastructure that others build their applications upon.

Data Conversion

Because the Synthesis engine implements the data format conversions and merging required for a SyncML server and makes that functionality available also to SyncML clients, it was easy to implement another fundamental change in SyncEvolution: 0.9 switches from an “all items are sent verbatim to the SyncML server” to a “parse and convert” data handling model. The argument for the former approach was that the SyncML server should be the only entity in the system which does data conversion. But faced with real-world constraints, the previous releases already had to deviate from this approach to accommodate for minor client/server incompatibilities and for vCard 2.1 support.

The main reason for going to full semantic conversion is vCalendar 1.0 support. Support by servers for iCalendar 2.0, the only format supported by 0.8.1, is often still incomplete or even non-existent. By doing the conversion on the client side, SyncEvolution is now able to synchronize events and tasks with a wider variety of servers.

It is still true that properties not supported by a server cannot be synchronized to other devices, so using a server with full iCalendar 2.0 support is recommended. But in contrast to 0.8.1, information that can only be stored locally is no longer lost when receiving an incomplete update from the SyncML server thanks to intelligent merging provided by the Synthesis engine. This depends on an accurate description of the server’s capabilities, which might not be provided by all of them – still needs to be tested in more detail.

What do these changes mean for related projects?

I look forward to continuing the cooperation with Funambol and ScheduleWorld on interoperability. The use of open standards for synchronization is going to benefit everyone involved because it allows users to synchronize who wouldn’t be able to do that otherwise, but it requires testing to ensure that the different components in the system really work well together. Other SyncML server developers and operators are welcome to join the SyncEvolution interoperability testing as long as they:

  • provide test accounts
  • assist with analyzing problems
  • support SyncEvolution users who want to use the service

With the goal of supporting direct synchronization, SyncEvolution enters a space that other projects also focus on, for example OpenSync and Conduit. We already discussed opportunities for cooperation with OpenSync: OpenSync currently doesn’t have stable code for conversion between different formats like vCalendar and iCalendar or vCard 2.1<->3.0, which is essential for a useful 0.40 release. Perhaps the Synthesis code can fill this gap. The Conduit project seems to be less focused on PIM data and mobile devices, but if there is a chance to cooperate, then it’s certainly worth discussing that.

The goal is to continue with SyncEvolution and Synthesis not just as open source, but also as open projects, with as much communication on public channels as possible. This is just getting started, so bear with us (and kindly remind us!) while we figure out how to do this properly.

SyncEvolution 0.9 beta 1

The focus of this first beta was to reach the same level of functionality and stability as in
0.8.1. Therefore this beta does not yet bring much new features; this
will be the focus of further beta releases until finally 0.9 will be a
full replacement for 0.8.1.

  • vCalendar 1.0 is now supported. Because this hasn’t been tested that much yet, events and tasks are still disabled in most of the default configurations (SF #2635973).
  • Both libcurl and libsoup can be selected at compile time as HTTP(S) transport mechanism.
  • SF #2101015: Expect: 100-continue header results in 417 Error with proxy. Should no longer occur with the HTTP transports in this release.
  • SF #1874805: Syncronization with Funambol results in loosing all-day property. This now works thanks to the Synthesis data conversion rules.
  • SF #2586600: Synchronization with mobical.net fails. Should work now because of the different SyncML implementation (untested).
  • SF #2542968: separator for categories should not be escaped. Done correctly by the Synthesis vcard conversion.

Known shortcomings in this release which will be fixed before the final 0.9:

  • Verbatim file backups of items on the SyncML server are currently not possible: the SyncEvolution “file” backend still exists, but all items are converted by the Synthesis engine and therefore must be in a format supported by the engine.
  • HTTPS can be used with libsoup, but certificate checking is always disabled. Need to find a portable way to determine where the certificate file is on various systems.
  • Log file handling is not yet unified: the traditional client.log contains only high-level SyncEvolution log entries. Low-level SyncML and engine log entries are in sysync_*.html files.
  • stdout and stderr messages from system libraries are visible on the console. 0.8.1 used to redirect those into the client.log to hide this noise; this will be added again. In the meantime, ignore messages like “Deadlock potential – avoiding evil bug!”. This is liborbit telling us that it is (hopefully successfully) handling something nasty.

Download + Installation

The source of SyncEvolution 0.9 beta 1 with the corresponding Synthesis source included is available here: http://www.estamos.de/download/syncevolution/sources/

Precompiled binaries will be made available for Moblin soon. For other platforms please compile from source. Compilation, installation and usage works as described in the README and on the upstream project site. Here’s how one can compile and use the latest sources:

cd /tmp
git clone git://git.moblin.org/syncevolution.git
cd syncevolution
./autogen.sh
mkdir compilation
cd compilation
../configure --disable-shared --with-synthesis-src=git://git.moblin.org/libsynthesis.git
make
PATH=`pwd`/src:$PATH
syncevolution

Further Information

Web sites:

The following mailing lists are used for discussions:

The main bug tracker for SyncEvolution is on bugzilla.moblin.org (query open issues, report a problem or request an enhancement).

SyncML related issues can be filed there or directly on synthesis.ch.

The code is available in git repositories:

  • SyncEvolution: git://git.moblin.org/syncevolution.git
  • upstream Synthesis: git://www.synthesis.ch/libsynthesis.git
  • staging area for patches against upstream Synthesis: git://git.moblin.org/libsynthesis.git

The goal of the libsynthesis repository on moblin.org is to prepare patches developed by the core SyncEvolution team and publish them for upstream inclusion. If you want to contribute patches yourself, then please work with the Synthesis repository and contact Synthesis regarding their contributor agreement.

{ 8 } Comments

  1. Callum | May 13, 2009 at 10:41 pm | Permalink

    Congrats on the move to working on your hobby full time. Must be a really great feeling. SyncEvolution is one of those little gems that I’d be lost without, but I barely notice. The best way for software to be. Thank you very much for your hard work to date and your continuing efforts in the future. :-)

  2. Jonas B. | May 13, 2009 at 10:41 pm | Permalink

    What about Opensync? Is it not a valid way forward? How does SyncEvolution differ from this?

  3. Patrick Ohly | May 14, 2009 at 9:17 am | Permalink

    Callum, thanks for your kind comments – “barely notice” is indeed how it should be.

    Jonas, the biggest difference between SyncEvolution+Synthesis and OpenSync is perhaps this: SyncEvolution+Synthesis is code which works for some of the things OpenSync tries to do and could be used to cover more of them (bottom up, regular usable releases). OpenSync started from the architecture and is still in the process of refining and implementing it, with key aspects undecided/unimplemented (e.g., conversion of calendar data between different formats) and no stable release at the moment.

    Both approaches are valid, although I prefer bottom up because one can test concepts earlier and gets to (intermediate) solutions faster. At the end of the day, it depends on the execution. We’ll see what comes out of OpenSync. They are welcome to use the Synthesis source code if it helps them (we already discussed some ideas with them), but this is not something I have time for myself.

  4. Martin Fick | May 22, 2009 at 7:55 pm | Permalink

    “The core engine is written in C++ and implements both the client *and* the server side of the synchronization. ”

    Are you referring to this synthesis server:
    http://www.synthesis.ch/prod_server.php?lang=e&lay=desk
    It appears to not be open source? Or is there an open source part that you were referring to?

    If there is a workable free/libre/open source alternative to funambol, I would really love to find it!

  5. Patrick Ohly | May 22, 2009 at 10:28 pm | Permalink

    Martin, not all parts of the Synthesis server that you mentioned have been open-sourced. But the key component, the engine itself, is completely open source now and thus could be integrated into a desktop sync solution.

    In the meantime I suggest that you give the Synthesis demo server a try. It’s as easy to run as the Funambol server and if you don’t like the data formats it supports out of the box, then you can change the XML configuration, which is the part which defines the data formats (see this LWN article).

  6. Martin Fick | May 22, 2009 at 11:01 pm | Permalink

    Thanks for the reply and suggestion. I really am looking for an open source solution though.

    “But the key component, the engine itself, is completely open source now and thus could be integrated into a desktop sync solution.”

    Hmm, do you know of anyone planning on doing this? What would this take, is it something worth attempting on my own, or is this a major project?

  7. Patrick Ohly | May 23, 2009 at 4:04 pm | Permalink

    Martin, after returning from my family time off in two weeks, I’ll start working on such a desktop integration of the Synthesis engine. If you want to help or ensure that your own wishes are taken into account, then I suggest that you subscribe to the SyncEvolution mailing list.

  8. François Rey | July 5, 2009 at 1:39 pm | Permalink

    Anyone interested in having a KDE centralized sync tool should vote for this wish:
    https://bugs.kde.org/show_bug.cgi?id=107681

    I just posted a comment there crosslinking to this page, hoping for some synergy…

{ 1 } Trackback

  1. [...] already started writing about my full-time job in this blog, so I might as well expand the scope even further and add a private announcement: [...]

Post a Comment

Your email is never published nor shared. Required fields are marked *