Sync won't work using Ca(l|rd)DAV-Sync in 17.1.003

[oscar]

Hi all,

Last week I upgraded my eGW instance from 1.8 to 17.1.003 by downloading the release from github (I can’t use a package manager since this instance is on a shared host).
I installed both egroupware-epl-17.1.20181018.zip and egroupware-epl-contrib-17.1.20181018.zip, upgrade went fine.

My good old MokobiSync did’t work anymore and since support stopped, I bought CalDAV-sync.
However, I can’t get to sync my calendar.
Log at eGW shows me:

*** 84.192.100.56 2018-11-16T17:00:16+01:00
OPTIONS /groupdav.php/(username)/calendar/ HTTP/1.1
Authorization: Basic *******************
Content-Length: 0
Host: (domainname)
Connection: Keep-Alive
User-Agent: CalDAV-Sync/0.4.32 (BullittGroupLimited; CatS60; Android 6.0.1; nl_NL; org.dmfs.caldav.lib/748; like iOS/5.0.1 (9A405) dataaccessd/1.0)

HTTP/1.1 200 OK
Date: Fri, 16 Nov 2018 16:00:16 GMT
Server: Apache/2
X-Powered-By: PHP/5.6.38
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-Dav-Powered-By: EGroupware 17.1.003 CalDAV/CardDAV/GroupDAV server
MS-Author-Via: DAV
X-WebDAV-Status: 200 OK
DAV: 1, 2, access-control, calendar-access, calendar-auto-schedule, calendar-proxy, calendarserver-principal-property-search, calendarserver-private-events, calendar-managed-attachments
Allow: OPTIONS, REPORT, PROPFIND, PROPPATCH, MKCOL, GET, POST, PUT, DELETE, COPY, MOVE, LOCK, UNLOCK, ACL, HEAD
Content-length: 0
*** OPTIONS /(username)/calendar/ --> "200 OK" took 0.000 s

*** 84.192.100.56 2018-11-16T17:00:17+01:00
PROPFIND /groupdav.php/(username)/calendar/ HTTP/1.1
Authorization: Basic *******************
Content-Type: application/xml; charset="utf-8"
Depth: 0
Content-Length: 522
Host: (domainname)
Connection: Keep-Alive
User-Agent: CalDAV-Sync/0.4.32 (BullittGroupLimited; CatS60; Android 6.0.1; nl_NL; org.dmfs.caldav.lib/748; like iOS/5.0.1 (9A405) dataaccessd/1.0)
Accept-Encoding: gzip

<?xml version='1.0' encoding='utf-8' ?><B:propfind xmlns:A="http://calendarserver.org/ns/" xmlns:B="DAV:" xmlns:C="http://apple.com/ns/ical/" xmlns:D="urn:ietf:params:xml:ns:caldav"><B:prop><B:supported-report-set /><C:refreshrate /><B:displayname /><A:source /><B:current-user-privilege-set /><D:managed-attachments-server-URL /><A:getctag /><D:max-attachments-per-resource /><C:calendar-color symbolic-color="custom" /><D:max-attachment-size /><D:supported-calendar-component-set /><B:sync-token /></B:prop></B:propfind>
HTTP/1.1 207 Multi-Status
Date: Fri, 16 Nov 2018 16:00:17 GMT
Server: Apache/2
X-Powered-By: PHP/5.6.38
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-Dav-Powered-By: EGroupware 17.1.003 CalDAV/CardDAV/GroupDAV server
X-WebDAV-Status: 207 Multi-Status
DAV: 1, 2, access-control, calendar-access, calendar-auto-schedule, calendar-proxy, calendarserver-principal-property-search, calendarserver-private-events, calendar-managed-attachments
Content-Type: text/xml; charset="utf-8"

<?xml version="1.0" encoding="utf-8"?>
<D:multistatus xmlns:D="DAV:">
 <D:response xmlns:ns0="urn:uuid:c2f41010-65b3-11d1-a29f-00aa00c14882/" xmlns:ns2="urn:ietf:params:xml:ns:caldav" xmlns:ns3="http://calendarserver.org/ns/" xmlns:ns4="http://apple.com/ns/ical/">
  <D:href>/groupdav.php/(username)/calendar/</D:href>
   <D:propstat>
    <D:prop>
     <D:displayname>Calendar (name)</D:displayname>
     <ns2:supported-calendar-component-set xmlns:ns2="urn:ietf:params:xml:ns:caldav"><ns2:comp name="VEVENT"/></ns2:supported-calendar-component-set>
     <D:supported-report-set><D:supported-report><D:report><ns2:calendar-query /></D:report></D:supported-report><D:supported-report><D:report><ns2:calendar-multiget /></D:report></D:supported-report><D:supported-report><D:report><ns2:free-busy-query /></D:report></D:supported-report></D:supported-report-set>
     <ns3:getctag>1542286474</ns3:getctag>
     <D:current-user-privilege-set><D:privilege><D:read-current-user-privilege-set /></D:privilege><D:privilege><D:read /></D:privilege><D:privilege><D:write-properties /></D:privilege><D:privilege><D:bind /></D:privilege><D:privilege><D:write-content /></D:privilege><D:privilege><D:unbind /></D:privilege><D:privilege><ns2:read-free-busy /></D:privilege></D:current-user-privilege-set>
    </D:prop>
   <D:status>HTTP/1.1 200 OK</D:status>
  </D:propstat>
   <D:propstat>
    <D:prop>
     <ns4:refreshrate/>
     <ns3:source/>
     <ns2:managed-attachments-server-URL/>
     <ns2:max-attachments-per-resource/>
     <ns4:calendar-color/>
     <ns2:max-attachment-size/>
     <D:sync-token/>
   </D:prop>
   <D:status>HTTP/1.1 404 Not Found</D:status>
  </D:propstat>
 </D:response>
</D:multistatus>
*** PROPFIND /(username)/calendar/ --> "207 Multi-Status" took 0.037 s

Syncing my contacts with CardDAV-Sync Free seems to work fine.

Any help here is appreciated, thanks in advance
Oscar

[StefanU]

Hi Oscar,

I can’t read anything from the logs.
Have you checked the preferences for CalDAV / DardDAV Sync?
As a result, when you call the URL (https://domainname.de/egroupware/groupdav.php/calendar) in the web browser, you should be able to see the appointments:

Greetings
Stefan

[oscar]

Hi Stefan,
Thanks for your response.
When I visit that URL from my browser I do see my calendar indeed, so that seems to work fine.
Perhaps it’s an issue with Google Calendar at my phone: when I try to make the calendar visible by ticking the checkbox, after a second or so the tick is removed ???

Is that something you’ve seen before?

Regards,
Oscar

[StefanU]

Hi Oscar,

Is that something you’ve seen before?

No, I don’t know that one. But I wouldn’t sync with an Android standard calendar either. There you don’t have your data under control…

You know that you can also use EGroupware as a web app? With a mobile phone from 5" this works quite well.

Is the log shown above from EGroupware?:

Which address have you entered exactly?
Would you have another smartphone to test?

Greetings
Stefan

[oscar]

Hi Steven

Yes, the log is from eGroupWare.
In the meantime I’m also trying to sync my contacts using CardDAV-Sync. At first I thought that worked; in eGW logging I see all my contacts being sent, but they don’t show up at my Android phone. For now this is a bigger issue than the Calendar (I tried other calendars as well, same result), so I wanna focus on contacts first.

I didn’t know about the web app option and when I try that, I see all names for all my contacts, but without any data (like phone numbers). Yet, on the browser interface all data is there.

In the CardDAV logging i see:

20181119T071805Z V org.dmfs.syncadapter.SyncAdapter background data setting: null  background data value:true
20181119T071805Z V org.dmfs.syncadapter.SyncAdapter network type: 1   newtork subtype: 0   is connected: true   is available: true
20181119T071805Z V org.dmfs.syncadapter.SyncAdapter starting sync ...
20181119T071805Z V org.dmfs.contacts.GroupManager performed group updates in: 14ms
20181119T071805Z V org.dmfs.carddav.syncadapter.SyncAdapter Syncing (url)/groupdav.php/(user)/addressbook/ (user)
20181119T071806Z V org.dmfs.dav.methods.DavOptions Code : 200
20181119T071806Z I org.dmfs.dav.Dav enabling preemptive Authentication
20181119T071806Z V org.dmfs.contacts.ContactsManager time to update uids: 44 ms
20181119T071806Z V org.dmfs.dav.CardDav sync-collection not supported - falling back to full sync
20181119T071806Z I org.dmfs.dav.CardDav ignoring non-contact result (url)/groupdav.php/(user)/addressbook/ with content-type: httpd/unix-directory
20181119T071806Z V org.dmfs.dav.CardDav found item: 39.vcf 0 39:17  text/vcard
20181119T071806Z V org.dmfs.dav.CardDav found item: 40.vcf 0 40:4  text/vcard
20181119T071807Z V org.dmfs.dav.CardDav found item: 41.vcf 0 41:4  text/vcard
20181119T071807Z V org.dmfs.dav.CardDav found item: 42.vcf 0 42:10  text/vcard
[...]

which looks ok so far, but then:

20181119T071807Z V org.dmfs.dav.CardDav found item: 314.vcf 0 314:0  text/vcard
20181119T071807Z V org.dmfs.dav.CardDav received 212 items
20181119T071807Z V org.dmfs.sync.Synchronator pairing entities
20181119T071807Z V org.dmfs.sync.Synchronator sorting 212 pairs
20181119T071807Z I org.dmfs.sync.Synchronator r/u/n: 0/0/212
20181119T071807Z V org.dmfs.vcardadapter.VCardSource preloading 40 entities
20181119T071807Z V org.dmfs.dav.CardDav multiget called
20181119T071807Z V org.dmfs.dav.methods.DavReport Status: 403
20181119T071807Z V org.dmfs.dav.methods.DavReport <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
20181119T071807Z V org.dmfs.dav.methods.DavReport <html><head>
20181119T071807Z V org.dmfs.dav.methods.DavReport <title>403 Forbidden</title>
20181119T071807Z V org.dmfs.dav.methods.DavReport </head><body>
20181119T071807Z V org.dmfs.dav.methods.DavReport <h1>Forbidden</h1>
20181119T071807Z V org.dmfs.dav.methods.DavReport <p>You don't have permission to access /groupdav.php/(user)/addressbook/
20181119T071807Z V org.dmfs.dav.methods.DavReport on this server.<br />
20181119T071807Z V org.dmfs.dav.methods.DavReport </p>
20181119T071807Z V org.dmfs.dav.methods.DavReport <p>Additionally, a 403 Forbidden
20181119T071807Z V org.dmfs.dav.methods.DavReport error was encountered while trying to use an ErrorDocument to handle the request.</p>
20181119T071807Z V org.dmfs.dav.methods.DavReport </body></html>
20181119T071807Z V org.dmfs.dav.methods.DavGet Code : 200
20181119T071807Z V org.dmfs.dav.methods.DavGet ETag : 314:0-gzip
20181119T071807Z V org.dmfs.dav.CardDav name not recognized in " � ������ E�MO�0 ��M� z�BK e �AE � `$z �%�Th���d������TQj賴�1�UӖF�� Fuc�2 ��X每�aq���n�}En I�]ܺ��|&\z�cL\F ��J� �î*3�4t'��ކ�5����4��$��Q�U�� F�)�7�T�tϵzx2{uߛ2Sp' ʈȘĂH���B:�8��!�Ÿ�� �2��n�� 1�# l�� F��!`<��S&;�a# / )���� K�!�" ��"
20181119T071807Z E org.dmfs.vcardadapter.VCardAdapter no vCard data returned for: (url)/groupdav.php/(user)/addressbook/314.vcf
20181119T071807Z V org.dmfs.dav.methods.DavGet Code : 200
20181119T071807Z V org.dmfs.dav.methods.DavGet ETag : 184:3-gzip
20181119T071807Z V org.dmfs.dav.CardDav name not recognized in " � ������ M��N�0 ��M� �k�-�H����"�-��b��f �b`7�} �E_L�5z��|�o�dF�0���U ���Eex� ֶR����u�w�׷���L��J�l���0l�� �أ !�d �eV�|� e!!Ы�(��n�/o+{� �� ��q�w/ �:�����u ���|��4�`i�<�� Aes>9�>�J]��s�Y�P (� #�� P�\k; _� #3�(�*#s ��xx��P%��r��8�R�D� �P���d*�C �n9#4�$�6�q �(� �5��ߵ� e ��"
20181119T071807Z E org.dmfs.vcardadapter.VCardAdapter no vCard data returned for: http://(domainname)/groupdav.php/(username)/addressbook/184.vcf
20181119T071807Z V org.dmfs.dav.methods.DavGet Code : 200
[...]

for all vcards.
Yet, opening the vcard via my browser (which opens xed) works well.
Same with an export from eGW; both a vcf file (full addressbook) as the full calendar (ics) look ok in xed, except for some blanks in phone numbers, e.g. “+32 485[…]” shows “+32Â 485[…]”. This happens with just a handful of addresses.
And the countryname “BELGIË” is shown as “BELGIË”

I don’t have another Android to test with. I’m gonna test with an iPhone later today; that one also lost all data since the 1.8 -> 17.1.003 upgrade but I haven’t looked into that yet.

Regards,
Oscar

[oscar]

iPhone has the same issues (tested with contacts only so far).
eGW logging sais:

(logging removed)

Yet, all existing contacts have been removed from the iPhone and the contactlist is empty.

[oscar]

In the meantime tested with Calendar at the iPhone too, same result: no errors and no data :-/

[oscar]

Hi guys,

Do you have any more suggestions for me?
I’ve been playing around with character sets, but still no result at all.

To summarize:

• It looks like eGW sends all contact data, no issues there
• Both in Android and iPhone the contacts don’t show up
• I don’t no where to check iPhone logs, but CardDAV-sync says: “name not recognized in " � ����[…]” followed by “no vCard data returned for: […]”
• I don’t see any errors when syncing the calendar data, but with the several calenders I tested, each time I tick an eGW to be visible, the tick is almost immediatly removed
• I can use my calendar with the eGW webapp, but not my contacts: I see all names, but in the eGW webapp, phone nrs, address info etc is missing.
  I do see al that info in eGW using the browser.

I’m stuck :-/

Thanks in advance,
Oscar

[oscar]

This might be something: from apache’s error_log:
[Wed Nov 21 09:56:18.630548 2018] [access_compat:error] [pid 3813:tid 140102151821056] [client 84.192.100.56:34310] AH01797: client denied by server configuration: (path)/groupdav.php

(I changed the groupdav.htaccess to the /egroupware it removed from the paths since this installation is in the root and copied it to .htaccess of course ;))

[StefanU]

Hi Oscar,

/home/oveas/domains/oveas.net/public_html/groupdav.php

… yncAdapter Syncing http://oveas.net/groupdav.php/oscar/addressbook/ oscar

I’m not sure that’s true in your case:
EGroupware must not be installed into the root directory of the web server.

Greetings
Stefan

[oscar]

Hi Stefan,

AFAIK that shouldn’t be a problem (hasn’t been for the last 10 years or so…), but I just moved it to /egroupware and nothing changed - still the same problems.

Greets,
Oscar

[oscar]

I still see this in my error_log:
AH01797: client denied by server configuration: (path)/egroupware/groupdav.php

[oscar]

Hmmmm, that’s from the error_log, my access_log is continuously being written with all GET accesses from my phone:

(logging removed)

[oscar]

I just recreated the account at CardDAV-Sync, just before I reanabled I saw another error: AH01071, followed by the AH01797 when the account became active:

[Wed Nov 21 16:12:22.085016 2018] [proxy_fcgi:error] [pid 22888:tid 140399762749184] [client (ip):58454] AH01071: Got error 'Primary script unknown\n'
[Wed Nov 21 16:12:23.436581 2018] [access_compat:error] [pid 22888:tid 140399897032448] [client 37.114.94.149:35090] AH01797: client denied by server configuration: (path)/egroupware/groupdav.php

[oscar]

No more suggestions?
The message “name not recognized in " � ����[…]” implies eGW is sending incorrect data… it has to be caused by something :-/

[oscar]

Solved!
I asked my hosting provider to reboot the server, after which the logmessage

AH01071: Got error 'Primary script unknown\n'

disappeared .
I still saw

`AH01797: client denied by server configuration: (path)/egroupware/groupdav.php`

but after adding this to the apache conf:

<Directory (path-to-egroupware)>
    Order allow,deny
    Allow from all
    Require all granted
</Directory>

both addressbook and calender could be synced.

[StefanU]

Hi Oscar.

Great, congratulations!

That’s why we don’t recommend this type of installation. This is hard for us to support.

Here is an example of what is usually written into the web server configuration in the package installation:
Apache:

EGroupware/egroupware/blob/c4e4eb45a4db870b0370a1b8700d7f8db216b7ee/doc/rpm-build/apache.conf

Nginx:

EGroupware/egroupware/blob/c4e4eb45a4db870b0370a1b8700d7f8db216b7ee/doc/rpm-build/nginx.conf

Greetings
Stefan