SOGo | BTS

View Issue Details Jump to Notes ] Related Changesets ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0004723SOGosogo-toolpublic2019-04-02 13:352019-05-23 15:09
Reporterjeouz 
Assigned Toludovic 
PriorityhighSeveritymajorReproducibilityalways
StatusresolvedResolutionfixed 
Platform[Server] LinuxOSUbuntuOS Version16.04 LTS
Product Version4.0.7 
Target VersionFixed in Version4.0.8 
Summary0004723: sogo-tool users unknown - backup not working
DescriptionHello,

I recently installed SOGo on my server and it's running like a charm! There is just that one bug with creating an backup.
Whenever I run sogo-tool backup I get the following message (for every user I got in my ldap directory):

2019-04-02 19:08:25.905 sogo-tool[16354:16354] user 'postmaster' unknown

I found this old bug (see https://sogo.nu/bugs/print_bug_page.php?bug_id=3782 [^]) but in my case all users are still existent in my ldap.

After a while of testing I managed to understand the bug.

Whenever I login with an user and run the backup afterwards it works as intended.
So if I logged in on all my testaccounts the backup was working fine.

The weird thing comes now, whenever I run the backup again after some time (I assume it's those 300 seconds cache cleanup interval) SOGo seems to forget about the users and gives me again the error that all users are unknown.

So every cron I made for the last few days just showed me that there is nothing to backup because no user existed.

I assume this behaviour is not intended.

One important thing might be my config (see my config under additional information). I basicly took the base config and applied the %d patch for just an domainwide-global addressbook (see https://sogo.nu/bugs/view.php?id=3685 [^]).

Any idea what this could be? I didn't change anything apart from your basic configuration and the %d domain fix for the adressbooks.

I guess it's not intended that SOGo loses the accounts after the cache is cleaned?

Really sorry for bothering you!
Steps To Reproduce1. bash sogo-tool backup all users. Result: users unknown
2. login into SOGo with an user
3. bash sogo-tool backup again. Result: user is now found and an backup got created
4. wait a few minutes (probably those 300seconds from cache cleanup interval) and run bash sogo-tool backup again. Result: users unknown.


Additional InformationConfig: https://pastebin.com/AYkQWyLB [^]
Tagsldap, sogo-tool
Attached Files

- Relationships Relation Graph ] Dependency Graph ]

-  Notes
(0013512)
jeouz (reporter)
2019-04-02 13:39

The complete Results are here: https://pastebin.com/dAPjyHRg [^]

If you need more details please tell me.

Thanks in forward.
User avatar (0013513)
francis (administrator)
2019-04-02 13:46

Try to append the domain to your username when calling sogo-tool (ex: postmaster@example.com).
(0013514)
jeouz (reporter)
2019-04-02 14:11

I run sogo-tool with the backup ALL option. This should take every user that is existent? Otherwise I would need to write an loop and go through all my domains and users individually?

But I tried running it just for one user. I still get the response:

sogo-tool backup /var/vmail/backup/test david@test.de
<0x0x563d33ac5520[SOGoCache]> Cache cleanup interval set every 300.000000 seconds
<0x0x563d33ac5520[SOGoCache]> Using host(s) '127.0.0.1' as server(s)
2019-04-02 20:08:12.485 sogo-tool[30880:30880] Creating NGLdapConnection instance for bindDN 'cn=vmail,dc=v22018112979977771,dc=ultrasrv,dc=de'
2019-04-02 20:08:12.495 sogo-tool[30880:30880] Creating NGLdapConnection instance for bindDN 'cn=vmail,dc=v22018112979977771,dc=ultrasrv,dc=de'
2019-04-02 20:08:12.497 sogo-tool[30880:30880] user 'david' unknown

When I log into SOGo with the user everything is working, as well as the backup:

sogo-tool backup /var/vmail/backup/test david@test.de
<0x0x55769876b520[SOGoCache]> Cache cleanup interval set every 300.000000 seconds
<0x0x55769876b520[SOGoCache]> Using host(s) '127.0.0.1' as server(s)
2019-04-02 20:10:01.879 sogo-tool[32414:32414] Creating NGLdapConnection instance for bindDN 'cn=vmail,dc=v22018112979977771,dc=ultrasrv,dc=de'

in /var/vmail/backup/test there is the david@test.de file.

Any other idea?
User avatar (0013515)
francis (administrator)
2019-04-02 14:23

Using sogo-tool and placeholders in your LDAP source is just not tested/working for now and specifying ALL will likely never work with a dynamic configuration (!).
(0013516)
jeouz (reporter)
2019-04-02 15:13

Thanks for the answer. I will write an script to loop through all domains and user then.
(0013517)
jeouz (reporter)
2019-04-02 15:58

Another quick update:

ldapsearch -D "cn=vmail,dc=v22018112979977771,dc=ultrasrv,dc=de" -W -x -LLL 'mail=*' | grep "mail:" | cut -d " " -f 2 | paste -sd " "

gives me basicly an list of all users in the format of user@domain.com.
I tried appending that list of users to sogo-tool but even with correcly specifying the correct mail, sogo-tool still says that there is no user?
User avatar (0013518)
francis (administrator)
2019-04-02 16:11

It's normal since sogo-tool has not been patched to work with domain placeholders.
(0013519)
jeouz (reporter)
2019-04-02 16:33

Thanks for the information. Is there any chance that there will be a patch in the newer future?
(0013521)
zhb (reporter)
2019-04-04 02:55

Dear @francis,

We (iRedMail team) sponsored the domain placeholder in LDAP baseDN, can we also have this backup function working?
(0013522)
jeouz (reporter)
2019-04-04 07:02

That would be a charm if the backup works, otherwise the placeholder feature is not really usable in a production system.
(0013537)
jeouz (reporter)
2019-04-09 10:44

Any update so far? I would appreciate to get any information about wether you will fix that bug or not.
I'm planning to use SOGo on a longterm base for a few of my projects.

Thanks in forward.
User avatar (0013539)
francis (administrator)
2019-04-09 10:46

It will be fixed but you'll need to specify the domain.
(0013540)
zhb (reporter)
2019-04-09 23:22

hi @francis,

Specifying domain name will bring sysadmin some workload. Is it possible to make it working without specifying the domain name?

Possible solutions:

- Get users from SOGo SQL database directly. If user account exists in LDAP, but no any data in SOGo, we don't need to backup its data (no data at all).
- Add a new parameter, e.g. "backupBaseDN = xxx", used to query all mail users for backup. This dn should not contain any placeholder and the query result should contain all mail users.
(0013549)
jeouz (reporter)
2019-04-14 09:30

This is great news.

@zhb is right, specifying the domain would force us to write additional scripts.
But it would be doable to loop through all domains tho.

Any information on when this will be done?

Thanks in foward!
User avatar (0013577)
ludovic (administrator)
2019-05-23 15:09

I pushed a fix.

If you want things to work perfectly you MUST set UIDFieldName to 'mail' - otherwise it makes no sense as the domain name won't extractable dynamically from the database.

- Related Changesets
sogo: master f57ff869
Timestamp: 2019-05-23 15:02:10
Author: ludovic
Details ] Diff ]
(fix) fixed domain placeholder issue when using sogo-tool (fixes 0004723)
mod - NEWS Diff ] File ]
mod - SoObjects/SOGo/LDAPSource.h Diff ] File ]
mod - SoObjects/SOGo/SOGoUserManager.m Diff ] File ]

- Issue History
Date Modified Username Field Change
2019-04-02 13:35 jeouz New Issue
2019-04-02 13:36 jeouz Tag Attached: ldap
2019-04-02 13:36 jeouz Tag Attached: sogo-tool
2019-04-02 13:39 jeouz Note Added: 0013512
2019-04-02 13:46 francis Note Added: 0013513
2019-04-02 14:11 jeouz Note Added: 0013514
2019-04-02 14:23 francis Note Added: 0013515
2019-04-02 15:13 jeouz Note Added: 0013516
2019-04-02 15:58 jeouz Note Added: 0013517
2019-04-02 16:11 francis Note Added: 0013518
2019-04-02 16:33 jeouz Note Added: 0013519
2019-04-04 02:55 zhb Note Added: 0013521
2019-04-04 07:02 jeouz Note Added: 0013522
2019-04-09 10:44 jeouz Note Added: 0013537
2019-04-09 10:46 francis Note Added: 0013539
2019-04-09 23:22 zhb Note Added: 0013540
2019-04-14 09:30 jeouz Note Added: 0013549
2019-05-23 15:05 ludovic Changeset attached => sogo master f57ff869
2019-05-23 15:05 ludovic Assigned To => ludovic
2019-05-23 15:05 ludovic Resolution open => fixed
2019-05-23 15:09 ludovic Note Added: 0013577
2019-05-23 15:09 ludovic Status new => resolved
2019-05-23 15:09 ludovic Fixed in Version => 4.0.8


Copyright © 2000 - 2019 MantisBT Team
Powered by Mantis Bugtracker