View Issue Details

IDProjectCategoryView StatusLast Update
0004723SOGosogo-toolpublic2021-02-25 05:21
Reporterjeouz Assigned Toludovic  
PriorityhighSeveritymajorReproducibilityalways
Status resolvedResolutionfixed 
Platform[Server] LinuxOSUbuntuOS Version16.04 LTS
Product Version4.0.7 
Fixed in Version4.0.8 
Summary0004723: sogo-tool users unknown - backup not working
Description

Hello,

I recently installed SOGo on my server and it's running like a charm! There is just that one bug with creating an backup.
Whenever I run sogo-tool backup I get the following message (for every user I got in my ldap directory):

2019-04-02 19:08:25.905 sogo-tool[16354:16354] user 'postmaster' unknown

I found this old bug (see https://sogo.nu/bugs/print_bug_page.php?bug_id=3782) but in my case all users are still existent in my ldap.

After a while of testing I managed to understand the bug.

Whenever I login with an user and run the backup afterwards it works as intended.
So if I logged in on all my testaccounts the backup was working fine.

The weird thing comes now, whenever I run the backup again after some time (I assume it's those 300 seconds cache cleanup interval) SOGo seems to forget about the users and gives me again the error that all users are unknown.

So every cron I made for the last few days just showed me that there is nothing to backup because no user existed.

I assume this behaviour is not intended.

One important thing might be my config (see my config under additional information). I basicly took the base config and applied the %d patch for just an domainwide-global addressbook (see https://sogo.nu/bugs/view.php?id=3685).

Any idea what this could be? I didn't change anything apart from your basic configuration and the %d domain fix for the adressbooks.

I guess it's not intended that SOGo loses the accounts after the cache is cleaned?

Really sorry for bothering you!

Steps To Reproduce
  1. bash sogo-tool backup all users. Result: users unknown
  2. login into SOGo with an user
  3. bash sogo-tool backup again. Result: user is now found and an backup got created
  4. wait a few minutes (probably those 300seconds from cache cleanup interval) and run bash sogo-tool backup again. Result: users unknown.
Additional Information

Config: https://pastebin.com/AYkQWyLB

Tagsldap, sogo-tool

Activities

jeouz

jeouz

2019-04-02 17:39

reporter   ~0013512

The complete Results are here: https://pastebin.com/dAPjyHRg

If you need more details please tell me.

Thanks in forward.

francis

francis

2019-04-02 17:46

administrator   ~0013513

Try to append the domain to your username when calling sogo-tool (ex: postmaster@example.com).

jeouz

jeouz

2019-04-02 18:11

reporter   ~0013514

I run sogo-tool with the backup ALL option. This should take every user that is existent? Otherwise I would need to write an loop and go through all my domains and users individually?

But I tried running it just for one user. I still get the response:

sogo-tool backup /var/vmail/backup/test david@test.de
<0x0x563d33ac5520[SOGoCache]> Cache cleanup interval set every 300.000000 seconds
<0x0x563d33ac5520[SOGoCache]> Using host(s) '127.0.0.1' as server(s)
2019-04-02 20:08:12.485 sogo-tool[30880:30880] Creating NGLdapConnection instance for bindDN 'cn=vmail,dc=v22018112979977771,dc=ultrasrv,dc=de'
2019-04-02 20:08:12.495 sogo-tool[30880:30880] Creating NGLdapConnection instance for bindDN 'cn=vmail,dc=v22018112979977771,dc=ultrasrv,dc=de'
2019-04-02 20:08:12.497 sogo-tool[30880:30880] user 'david' unknown

When I log into SOGo with the user everything is working, as well as the backup:

sogo-tool backup /var/vmail/backup/test david@test.de
<0x0x55769876b520[SOGoCache]> Cache cleanup interval set every 300.000000 seconds
<0x0x55769876b520[SOGoCache]> Using host(s) '127.0.0.1' as server(s)
2019-04-02 20:10:01.879 sogo-tool[32414:32414] Creating NGLdapConnection instance for bindDN 'cn=vmail,dc=v22018112979977771,dc=ultrasrv,dc=de'

in /var/vmail/backup/test there is the david@test.de file.

Any other idea?

francis

francis

2019-04-02 18:23

administrator   ~0013515

Using sogo-tool and placeholders in your LDAP source is just not tested/working for now and specifying ALL will likely never work with a dynamic configuration (!).

jeouz

jeouz

2019-04-02 19:13

reporter   ~0013516

Thanks for the answer. I will write an script to loop through all domains and user then.

jeouz

jeouz

2019-04-02 19:58

reporter   ~0013517

Another quick update:

ldapsearch -D "cn=vmail,dc=v22018112979977771,dc=ultrasrv,dc=de" -W -x -LLL 'mail=*' | grep "mail:" | cut -d " " -f 2 | paste -sd " "

gives me basicly an list of all users in the format of user@domain.com.
I tried appending that list of users to sogo-tool but even with correcly specifying the correct mail, sogo-tool still says that there is no user?

francis

francis

2019-04-02 20:11

administrator   ~0013518

It's normal since sogo-tool has not been patched to work with domain placeholders.

jeouz

jeouz

2019-04-02 20:33

reporter   ~0013519

Thanks for the information. Is there any chance that there will be a patch in the newer future?

zhb

zhb

2019-04-04 06:55

reporter   ~0013521

Dear @francis,

We (iRedMail team) sponsored the domain placeholder in LDAP baseDN, can we also have this backup function working?

jeouz

jeouz

2019-04-04 11:02

reporter   ~0013522

That would be a charm if the backup works, otherwise the placeholder feature is not really usable in a production system.

jeouz

jeouz

2019-04-09 14:44

reporter   ~0013537

Any update so far? I would appreciate to get any information about wether you will fix that bug or not.
I'm planning to use SOGo on a longterm base for a few of my projects.

Thanks in forward.

francis

francis

2019-04-09 14:46

administrator   ~0013539

It will be fixed but you'll need to specify the domain.

zhb

zhb

2019-04-10 03:22

reporter   ~0013540

hi @francis,

Specifying domain name will bring sysadmin some workload. Is it possible to make it working without specifying the domain name?

Possible solutions:

  • Get users from SOGo SQL database directly. If user account exists in LDAP, but no any data in SOGo, we don't need to backup its data (no data at all).
  • Add a new parameter, e.g. "backupBaseDN = xxx", used to query all mail users for backup. This dn should not contain any placeholder and the query result should contain all mail users.
jeouz

jeouz

2019-04-14 13:30

reporter   ~0013549

This is great news.

@zhb is right, specifying the domain would force us to write additional scripts.
But it would be doable to loop through all domains tho.

Any information on when this will be done?

Thanks in foward!

ludovic

ludovic

2019-05-23 19:09

administrator   ~0013577

I pushed a fix.

If you want things to work perfectly you MUST set UIDFieldName to 'mail' - otherwise it makes no sense as the domain name won't extractable dynamically from the database.

Related Changesets

sogo: master f57ff869

2019-05-23 15:02

ludovic


Details Diff
(fix) fixed domain placeholder issue when using sogo-tool (fixes 0004723) Affected Issues
0004723
mod - NEWS Diff File
mod - SoObjects/SOGo/LDAPSource.h Diff File
mod - SoObjects/SOGo/SOGoUserManager.m Diff File

Issue History

Date Modified Username Field Change
2019-04-02 17:35 jeouz New Issue
2019-04-02 17:36 jeouz Tag Attached: ldap
2019-04-02 17:36 jeouz Tag Attached: sogo-tool
2019-04-02 17:39 jeouz Note Added: 0013512
2019-04-02 17:46 francis Note Added: 0013513
2019-04-02 18:11 jeouz Note Added: 0013514
2019-04-02 18:23 francis Note Added: 0013515
2019-04-02 19:13 jeouz Note Added: 0013516
2019-04-02 19:58 jeouz Note Added: 0013517
2019-04-02 20:11 francis Note Added: 0013518
2019-04-02 20:33 jeouz Note Added: 0013519
2019-04-04 06:55 zhb Note Added: 0013521
2019-04-04 11:02 jeouz Note Added: 0013522
2019-04-09 14:44 jeouz Note Added: 0013537
2019-04-09 14:46 francis Note Added: 0013539
2019-04-10 03:22 zhb Note Added: 0013540
2019-04-14 13:30 jeouz Note Added: 0013549
2019-05-23 19:05 ludovic Changeset attached => sogo master f57ff869
2019-05-23 19:05 ludovic Assigned To => ludovic
2019-05-23 19:05 ludovic Resolution open => fixed
2019-05-23 19:09 ludovic Note Added: 0013577
2019-05-23 19:09 ludovic Status new => resolved
2019-05-23 19:09 ludovic Fixed in Version => 4.0.8