Scalable OGo (SOGo)

View Issue Details Jump to Notes ] Related Changesets ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0003988SOGoBackend Calendarpublic2017-01-10 08:432017-06-24 05:59
ReporterMarcel 
Assigned Toludovic 
PrioritynormalSeverityminorReproducibilityrandom
StatusnewResolutionfixed 
Platform[Server] LinuxOSUbuntuOS Version14.04 LTS
Product Versionnightly v3 
Target VersionFixed in Version 
Summary0003988: Subscription to calendars deleted when database is unreachable
DescriptionThe Postgres database server storing the SOGo databases was overloaded (due to some other processes).

When a user with incoming shared calendars would try to CalDAV sync a calendar, if the database was unavailable at that time, it would delete the subscription to that calendar (if the database was reachable again for the deletion).

In the end, after some time of syncing and some time of heavily loaded database server, about half of the dozen+ subscriptions of one user had disappeared.
Steps To ReproduceLoad your database server heavily.

Sync from a CalDAV client (Evolution, in this case, but I doubt it makes a difference).

Enjoy the disappearing subscriptions.
Additional InformationI assume if the query for the calendar fails, the result is the same for "calendar not found" and "could not connect to the database". The wrapping code does the processing for "calendar not found" (i.e., delete the subscription), even if the result was "could not connect to the database".
TagsNo tags attached.
Attached Filespng file icon erreur-sogo.png [^] (7,803 bytes) 2017-06-20 09:23

- Relationships Relation Graph ] Dependency Graph ]

-  Notes
(0011494)
hkunz (reporter)
2017-03-15 11:55

I can confirm this (debian jessie).

After a postgres update (and an involved stopping/starting of the database) we "randomly" lost calendar subscriptions. I was puzzled until I found this bug report, which perfectly explains what happened.

I think this bug should be marked as (very) important, because (configuration) data is lost.
(0011497)
robert.k (reporter)
2017-03-16 03:10

we have something similar with Debian Jessie and Outlook 2013 with EAS
Subscriptions of calendar are lost.

The reason for connection lost was to many sql connections were opened so there was no more connection to database possible.

The only way to fix this was to create a new outlook Profile.

In Thunderbird we didn't noticed that until now.
(0011535)
robert.k (reporter)
2017-03-20 03:44
edited on: 2017-03-20 05:14

We had an delete of subcribed calendars in SOGoWeb too.

Our Problem are too many database connections (see bug https://sogo.nu/bugs/view.php?id=4083 [^]) so we have the delete more then one time at a user..

(0011548)
Mathias Roland (reporter)
2017-03-22 04:51

I can confirm this problem too.

The connection to the database server was lost due to a database server restart. After this one or more calendar subscriptions were lost by all users who tried to access the web calendar in the outage time.
(0011561)
alaniyatm (reporter)
2017-03-23 05:09

I can confirm this problem too.
Thunderbird with SOGo plugins disables calendars when DB is out of service.

For us this issue is wery important - 15k users are affected time to time...
User avatar (0011785)
francis (administrator)
2017-05-03 11:47

Is this problem only observed with a PostgreSQL database?
(0011786)
robert.k (reporter)
2017-05-04 01:26

We have the problem also and we have an mysql DB MySQL 5.5.54
User avatar (0011790)
ludovic (administrator)
2017-05-08 10:31

I've tried to reproduce it with no luck on MySQL.
(0011988)
realynot (reporter)
2017-06-19 05:53
edited on: 2017-06-20 09:23

Same problem with postgres 9.3, is very important for us to.
i tried to up max connection on postgres, but same problem...

edit : upload error file

User avatar (0011994)
ludovic (administrator)
2017-06-20 09:42

I need to have a reproducible way to duplicate the problem here.

We did extensive testing and never been able to trigger it.
(0011997)
realynot (reporter)
2017-06-22 04:32

At the moment we detect the loss of calendars with the command
tail -f sogo.log | grep PROPFIND | grep 404 | grep Calendar

And when calendar are lost :

Jun 22 07:38:18 sogod [6983]: 127.0.0.1 "PROPFIND /SOGo/dav/delxxx/Calendar/valxxx_3C8D-5790D500-D-6867F280/ HTTP/1.0" 404 87/394 0.010 - - 0
Jun 22 07:49:03 sogod [7003]: 127.0.0.1 "PROPFIND /SOGo/dav/delxxx/Calendar/valxxx_3C8D-5790D500-D-6867F280/ HTTP/1.0" 404 87/394 0.009 - - 0
Jun 22 08:04:11 sogod [7008]: 127.0.0.1 "PROPFIND /SOGo/dav/delxxx/Calendar/valxxx_3C8D-5790D500-D-6867F280/ HTTP/1.0" 404 87/394 0.009 - - 0
Jun 22 08:19:09 sogod [6989]: 127.0.0.1 "PROPFIND /SOGo/dav/delxxx/Calendar/valxxx_3C8D-5790D500-D-6867F280/ HTTP/1.0" 404 87/394 0.008 - - 0
Jun 22 09:55:18 sogod [6997]: 127.0.0.1 "PROPFIND /SOGo/dav/farxxx/Calendar/desxxx_personal/ HTTP/1

How can you help you see more clearly?

I will try to reinstall a full server with postgresql 9.6 instead of 9.3 ... no?
(0011998)
infrasysteme (reporter)
2017-06-22 05:55

Hi,

we have the same problem here since the last version of SOGo v2 (2.3.21).
For us, we didn't have this issue with the 2.3.20 (i will confirm that because we are actually downgrading SOGo to 2.3.20).
We use postgreSQL 9.3 and we will try to upgrade to postgresql 9.6.

Also, we have the same logs that realynot when calendar are lost.

thanks for your help,
User avatar (0012000)
ludovic (administrator)
2017-06-22 08:27

Well if a PROPFIND returns a 404, that means the resource is gone from SOGo's point of view. there must be a reason why we say so if in fact, it isn't.
(0012001)
hkunz (reporter)
2017-06-22 09:08

This is just a wild guess.
We are seeing this problem since this bug has been fixed:
https://sogo.nu/bugs/view.php?id=4010 [^]

although this could also just be a coincidence.

Best,
Hp
User avatar (0012002)
ludovic (administrator)
2017-06-22 10:22

I've pushed a tentative fix, try it.
(0012003)
realynot (reporter)
2017-06-24 05:59

same problem with the fix :(

More and more users calendars are affected

- Related Changesets
sogo: master 7f00afce
Timestamp: 2017-06-22 10:20:54
Author: ludovic
Details ] Diff ]
(fix) check cache before using it (fixes 0003988)
mod - SoObjects/SOGo/SOGoGCSFolder.m Diff ] File ]
sogo: v2 a30efc6c
Timestamp: 2017-06-22 10:20:54
Author: ludovic
Details ] Diff ]
(fix) check cache before using it (fixes 0003988)
mod - SoObjects/SOGo/SOGoGCSFolder.m Diff ] File ]

- Issue History
Date Modified Username Field Change
2017-01-10 08:43 Marcel New Issue
2017-03-15 11:55 hkunz Note Added: 0011494
2017-03-16 03:10 robert.k Note Added: 0011497
2017-03-20 03:44 robert.k Note Added: 0011535
2017-03-20 04:53 robert.k Note Edited: 0011535 View Revisions
2017-03-20 05:14 robert.k Note Edited: 0011535 View Revisions
2017-03-22 04:51 Mathias Roland Note Added: 0011548
2017-03-23 05:09 alaniyatm Note Added: 0011561
2017-05-03 11:47 francis Note Added: 0011785
2017-05-04 01:26 robert.k Note Added: 0011786
2017-05-08 10:31 ludovic Note Added: 0011790
2017-06-19 05:53 realynot Note Added: 0011988
2017-06-20 09:23 realynot File Added: erreur-sogo.png
2017-06-20 09:23 realynot Note Edited: 0011988 View Revisions
2017-06-20 09:42 ludovic Note Added: 0011994
2017-06-22 04:32 realynot Note Added: 0011997
2017-06-22 05:55 infrasysteme Note Added: 0011998
2017-06-22 08:27 ludovic Note Added: 0012000
2017-06-22 09:08 hkunz Note Added: 0012001
2017-06-22 10:22 ludovic Changeset attached => sogo master 7f00afce
2017-06-22 10:22 ludovic Assigned To => ludovic
2017-06-22 10:22 ludovic Resolution open => fixed
2017-06-22 10:22 ludovic Changeset attached => sogo v2 a30efc6c
2017-06-22 10:22 ludovic Note Added: 0012002
2017-06-24 05:59 realynot Note Added: 0012003


Copyright © 2000 - 2017 MantisBT Team
Powered by Mantis Bugtracker