View Issue Details

IDProjectCategoryView StatusLast Update
0005078SOGoBackend Mailpublic2021-02-26 05:36
Reporterzhb Assigned Tofrancis  
PriorityhighSeveritymajorReproducibilityalways
Status resolvedResolutionfixed 
Product Version4.3.2 
Fixed in Version5.0.0 
Summary0005078: Does the latest nightly build support STARTTLS and SSL for SMTP service?
Description

Dear developers,

According to issue 31[1], STARTTLS and SSL support for smtp service had been merged to SOPE and SOGO projects. i tried to enable STARTTLS support with latest nightly build (4.3.2.20200709):

SOGoMailingMechanism = smtp;
SOGoSMTPServer = "smtp://127.0.0.1:587/?TLS=YES";
SOGoSMTPAuthenticationType = PLAIN;

but SOGo raises error message on web UI: "cannot send message: (smtp) authentication failure".
With debug enabled in Postfix, seems SOGo didn't perform STARTTLS command at all:

##################
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: connect from localhost[127.0.0.1]
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: smtp_stream_setup: maxtime=300 enable_deadline=0
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: match_hostname: localhost ~? 127.0.0.1
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: match_hostaddr: 127.0.0.1 ~? 127.0.0.1
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: > localhost[127.0.0.1]: 220 mail.deeptree.tech ESMTP Postfix
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: watchdog_pat: 0x55970f4eeeb0
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: < localhost[127.0.0.1]: EHLO localhost
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: dict_pcre_lookup: /etc/postfix/command_filter.pcre: EHLO localhost
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: match_list_match: localhost: no match
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: match_list_match: 127.0.0.1: no match
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: > localhost[127.0.0.1]: 250-mail.deeptree.tech
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: > localhost[127.0.0.1]: 250-PIPELINING
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: > localhost[127.0.0.1]: 250-SIZE 104857600
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: > localhost[127.0.0.1]: 250-ETRN
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: > localhost[127.0.0.1]: 250-STARTTLS
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: > localhost[127.0.0.1]: 250-ENHANCEDSTATUSCODES
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: > localhost[127.0.0.1]: 250-8BITMIME
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: > localhost[127.0.0.1]: 250 DSN
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: watchdog_pat: 0x55970f4eeeb0
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: < localhost[127.0.0.1]: QUIT
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: dict_pcre_lookup: /etc/postfix/command_filter.pcre: QUIT
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: > localhost[127.0.0.1]: 221 2.0.0 Bye
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: match_hostname: localhost ~? 127.0.0.1
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: match_hostaddr: 127.0.0.1 ~? 127.0.0.1
Jul 9 08:24:05 mail postfix/submission/smtpd[18095]: disconnect from localhost[127.0.0.1]
####################

Software versions:

sope49-gdl1-contentstore-4.3.2.20200709-1.el7.x86_64
sope49-appserver-4.9-20200708_1664.el7.1.x86_64
sope49-core-4.9-20200708_1664.el7.1.x86_64
sope49-mime-4.9-20200708_1664.el7.1.x86_64
sope49-xml-4.9-20200708_1664.el7.1.x86_64
sope49-ldap-4.9-20200708_1664.el7.1.x86_64
sope49-cards-4.3.2.20200709-1.el7.x86_64
sope49-gdl1-postgresql-4.9-20200708_1664.el7.1.x86_64
sope49-sbjson-2.3.1-20200708_1664.el7.1.x86_64
sope49-gdl1-4.9-20200708_1664.el7.1.x86_64
sogo-tool-4.3.2.20200709-1.el7.x86_64
sogo-4.3.2.20200709-1.el7.x86_64
sogo-ealarms-notify-4.3.2.20200709-1.el7.x86_64
sogo-activesync-4.3.2.20200709-1.el7.x86_64

[1] https://sogo.nu/bugs/view.php?id=31

TagsNo tags attached.

Relationships

related to 0005079 resolvedfrancis How to disable SSL cert verification for SMTP (auth) connection? 

Activities

the_nic

the_nic

2020-07-09 16:53

reporter   ~0014487

Yes, these changes are implemented. However, you probably won't get a valid certificate for 127.0.0.1, please specify the fqdn instead (or disable tls)

zhb

zhb

2020-07-09 16:57

reporter   ~0014489

Is it possible to disable cert validation?

zhb

zhb

2020-07-09 16:58

reporter   ~0014490

I understand it's the best to validate cert, but in testing server, we still need to test TLS/SSL support.

the_nic

the_nic

2020-07-10 08:13

reporter   ~0014509

Currently, this is not possible. You may, however set up your own CA, add it to your root store and then sign a certificate for your endpoint with your CA.

francis

francis

2020-07-10 14:24

administrator   ~0014514

There's no point in enabling TLS/SSL if you don't want the certificate to be verified. Read comments in related tickets.

zhb

zhb

2020-07-10 15:40

reporter   ~0014517

@francis I don't agree that.

  • We use port 25 for incoming emails, and 587 with TLS for submission. This is pretty normal.
  • If i don't use 587+TLS, i have to set SOGoSMTPServer to something like "127.0.0.1", or "smtp://127.0.0.1". But sogo will connect to port 25 for sending and this obviously causes error because port 25 doesn't support smtp auth.

If ssl cert verification is enforced, sogo can not send email with SOGoSMTPServer set to "smtp://127.0.0.1/?tls=YES" or "127.0.0.1", or "smtp://127.0.0.1".
Connection from/to localhost is considered as secure, so ssl cert verification can be disabled in this case. It doesn't have to be a FQDN smtp server address with a valid ssl cert (sure i understand a cert is the best choice).

zhb

zhb

2020-07-10 15:54

reporter   ~0014518

This commit mentions "specify fqdn requirement for SSL/TLS", but it doesn't solve the issue with IP address + TLS/SSL: https://github.com/inverse-inc/sogo/commit/a91a00e33cea3e365d8c43b4a9b15a9d55479afb
Note: just to make it clear, i clearly understand we need a valid cert if the request goes to a remote server, but my request is: we should have the option to disable ssl cert verification. Especially with "127.0.0.1" as smtp server address.

the_nic

the_nic

2020-07-13 12:32

reporter   ~0014521

@zhb But the certificate is signed with some domain in the subject, right? For now, you could just set that particular domain in your /etc/hosts:

127.0.0.1 some.mail.domain

And then set smtps://some.mail.domain to your sogo configuration. some.mail.domain will always resolve to localhost and validation should succeed.

zhb

zhb

2020-07-13 15:37

reporter   ~0014522

ok, let me ask in another way: Any plan to support disabling cert validation?

the_nic

the_nic

2020-07-13 15:53

reporter   ~0014523

I started looking into this quickly on https://github.com/the-nic/sope/tree/optional-tls-validation, but havent really continued. Note that I am not associated with inverse in any way, I'm doing this in my free time, so dont expect any particular support. I have also shown multiple ways how you may fix your setup.

You can still help with the implementation, though ;-)

zhb

zhb

2020-07-13 16:04

reporter   ~0014525

Dear @the_nic,

Thanks for your contribution. :)

I understand you already offered few suggestions, but disabling cert validation is quite common, not just in a quick testing environment, but also on production (connect to 127.0.0.1 instead of FQDN, of course we can argue this, but why disallow sysadmin to use 127.0.0.1?).

zhb

zhb

2020-07-25 05:56

reporter   ~0014583

Dear @the_nic,

Are you still working on the optional ssl validation feature? :)

the_nic

the_nic

2020-07-25 10:27

reporter   ~0014584

@zhb, I havent looked into it again, yet.

Why dont the other solutions work for you, by the way?

zhb

zhb

2020-07-25 10:52

reporter   ~0014585

My free + open source project "iRedMail" installs and configure SOGo for users, at initial installation stage, the installer doesn't know whether the server has a valid ssl cert, so usually we use "127.0.0.1" as IMAP/SMTP/Sieve server address, it's localhost and it's considered as secure, so we prefer to stick to "127.0.0.1" in the installer.

With default iRedMail configuration, port 25 is used to accept incoming emails, and 587 (TLS) is used for submission, in this case if SOGo supports TLS without verification ssl cert (again, cert for host "127.0.0.1" in our case) that would be the best for us, no additional (insecure) smtp port for SOGo without TLS/SSL.

If sysadmin gets a valid ssl cert, it's ok to replace 127.0.0.1 by the server hostname as server address. But this is not handled by iRedMail installer, so it's not an option for us.

the_nic

the_nic

2020-07-25 11:32

reporter   ~0014586

So maybe an option like AllowInsecureLocalhost would be more appropiate. With that option given, we could disable peer verification forlocalhost, 127.0.0.1

zhb

zhb

2020-07-25 14:35

reporter   ~0014587

I think a parameter in the URI might be better. For example:

smtp://127.0.0.1:587/?tls=YES&verify_cert=NO

  • No new parameter in sogo.conf introduced.
  • Standard DB URI format.
the_nic

the_nic

2020-08-02 13:05

reporter   ~0014606

SOPE: https://github.com/inverse-inc/sope/pull/58
SOGo: https://github.com/inverse-inc/sogo/pull/286

This will allow you to disable TLS validation if the remote is a localhost (localhost, 127.0.0.1/8, ...) : smtp://127.0.0.1:587/?tls=YES&tlsVerifyMode=allowInsecureLocalhost

zhb

zhb

2020-08-02 14:04

reporter   ~0014607

The value "allowInsecureLocalhost" is too weird.

  • Does it mean that ssl cert verification can be disabled for only "localhost"-like addresses? How about a LAN IP like 192.168.1.1?
  • Why not simply use "tlsVerify=NO" (or something else like "tlsVerifyCert=NO") to disable the verification? Why it's "allow insecure localhost"?
zhb

zhb

2020-08-02 14:06

reporter   ~0014608

Forgot to say "THANK YOU" for the contribution. :)

the_nic

the_nic

2020-08-02 17:07

reporter   ~0014609

Does it mean that ssl cert verification can be disabled for only "localhost"-like addresses?

Exactly that. It follows the same features like allow-insecure-localhost in chrome. So, you can use 127.0.0.1, localhost, and similar, including any other loopback address.

How about a LAN IP like 192.168.1.1?

This is really not intended for that. For that, you should really run a PKI. Please do not decrease security by default for your clients by disabling TLS validation. That is really bad behavior.

Also, according your description you'd need your clients to install a valid certificate for the smtp/imap servers anyways before your setup is going to work. Make them fill up the real host name in the SOGo config, too.

zhb

zhb

2020-08-03 14:34

reporter   ~0014611

I understand the security concerns, but it's better giving sysadmins such option. For example, connecting to an IMAP server in DMZ or a fully trusted LAN network.

the_nic

the_nic

2020-08-03 15:24

reporter   ~0014612

As a system administrator with root access you already have plenty of options to dilute the security of the system as needed.

zhb

zhb

2020-08-03 15:41

reporter   ~0014613

OK, i don't want to go further and currently this option is enough for a standalone mail server when IMAP is running on same host.
But i still suggest you consider my suggestion. :) If you don't want to support this feature, it's fine.

Thank you very much for the help, and hope SOGo team will merge it soon. :)

the_nic

the_nic

2020-08-03 15:43

reporter   ~0014614

Yes, it is even there (you'll find it in the code), I just want to leave it undocumented, as people tend to enable these options without understanding the consequences or fixing the issue properly.

francis

francis

2020-08-05 19:22

administrator   ~0014640

Pull requests were merged. Thanks @the_nic!

zhb

zhb

2020-08-11 08:43

reporter   ~0014663

Dear @the_nic,

tlsVerifyMode=allowInsecureLocalhost works fine if the server address is localhost, but not work with address 127.0.0.1 (didn't test IPv6 address [::1] yet). Could you help improve the code to support 127.0.0.1?

the_nic

the_nic

2020-08-11 08:52

reporter   ~0014664

Hey @zhb, thanks for the feedback. Interestingly, I added a test for localhost ips: https://github.com/inverse-inc/sogo/blob/master/Tests/Unit/TestNGInternetSocketAddress.m#L35

IPv6 does not work, at all. SOPE does not support ipv6 addresses, unfortunately.

zhb

zhb

2020-08-11 11:28

reporter   ~0014666

The problem is, "127.0.0.1" doesn't work. :(

the_nic

the_nic

2020-08-11 17:49

reporter   ~0014668

@zhb i have just tried with imap, and it works. Can you tell me which one (SMTP/IMAP/sieve) does not work with 127.0.0.1 and paste the config lines, please?

zhb

zhb

2020-08-12 01:01

reporter   ~0014681

Dear @the_nic,

Yes IMAP works, failed one is SMTP service. I didn't test managesieve yet, it was rush last time i test it. sorry.

the_nic

the_nic

2020-08-12 06:44

reporter   ~0014682

@zhb As said before, please share your config. It really sounds like an error on your end, like not setting the SMTP port 587 explicitly.

zhb

zhb

2020-08-12 08:37

reporter   ~0014690

Interesting, this time it works with either allowInsecureLocalhost or none.
I think we're fine now. :)

zhb

zhb

2020-08-19 06:19

reporter   ~0014704

Hi @the_nic

It's very weird that sometimes it works but sometimes not.

sogo.conf:

SOGoSMTPServer = &quot;smtp://127.0.0.1:587/?tls=YES&tlsVerifyMode=allowInsecureLocalhost&quot;;
SOGoMailingMechanism = smtp;
//SOGoSMTPAuthenticationType = PLAIN;

Postfix /etc/postfix/master.cf:

submission inet n       -       n       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=encrypt
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
  ...
  • Tried to send email on SOGo web UI, failed with error message Cannot send message: all recipients are invalid..
  • Postfix log:
Aug 18 23:16:41 mail postfix/submission/smtpd[41420]: NOQUEUE: reject: RCPT from myhostname[127.0.0.1]: 554 5.7.1 &lt;myhostname[127.0.0.1]>: Client host rejected: Access denied; from=&lt;my-email> to=&lt;external-gmail-email> proto=ESMTP helo=&lt;myhostname>

It says "Access denied", although it looks like an ACL issue, but we have permit_sasl_authenticated in Postfix submission, so if client performed TLS and SMTP SASL AUTH, it should be passed.

Tried using localhost as smtp server address in sogo.conf (SOGoSMTPServer = &quot;smtp://localhost:587/?tls=YES&tlsVerifyMode=allowInsecureLocalhost&quot;;), failed with same error.

zhb

zhb

2020-08-19 06:21

reporter   ~0014705

Forgot to mention OS and SOGo versions:

  • OS: 18.04.5 LTS (Bionic Beaver)
  • SOGo nightly build: 5.0.0.20200818-1
  • SOPE: 4.9.r1664.20200815
zhb

zhb

2020-08-19 07:05

reporter   ~0014708

Here's the debug log with ImapDebugEnabled = YES. Seems it didn't perform smtp sasl auth at all.

C: connect to &lt;0x0x558d77821170[NGInternetSocketAddress]: host=localhost not-filled>
S: &lt;SMTP-Reply: code=220 line='&lt;myhostname> ESMTP Postfix'>
C: EHLO &lt;myhostname>
S: &lt;SMTP-Reply: code=250 line='SMTPUTF8'>
S: pipelining extension supported.
S: size extension supported.
S: starttls extension supported.
C: STARTTLS
2020-08-18 23:59:38.467 sogod[54712:54712] SMTP: STARTTLS successfully performed
C: EHLO &lt;myhostname>
S: &lt;SMTP-Reply: code=250 line='SMTPUTF8'>
C: MAIL FROM:&lt;my-email>
C: RCPT TO:&lt;external-gmail-email>
Aug 18 23:59:38 sogod [54712]: &lt;0x0x558d778fe380[SOGoMailer]> error with recipient '&lt;external-gmail-email>'
C: QUIT
S: &lt;SMTP-Reply: code=221 line='2.0.0 Bye'>
the_nic

the_nic

2020-08-19 07:08

reporter   ~0014709

why is SOGoSMTPAuthenticationType commented out?

zhb

zhb

2020-08-19 07:20

reporter   ~0014711

That's the cause. I thought when SOGoMailingMechanism = smtp, the SOGoSMTPAuthenticationType = PLAIN is enabled by default.
Thanks @the_nic. :)

It's ok to close this now.

Issue History

Date Modified Username Field Change
2020-07-09 16:40 zhb New Issue
2020-07-09 16:53 the_nic Note Added: 0014487
2020-07-09 16:57 zhb Note Added: 0014489
2020-07-09 16:58 zhb Note Added: 0014490
2020-07-10 08:13 the_nic Note Added: 0014509
2020-07-10 14:10 francis Relationship added related to 0005079
2020-07-10 14:24 francis Assigned To => francis
2020-07-10 14:24 francis Status new => closed
2020-07-10 14:24 francis Resolution open => won't fix
2020-07-10 14:24 francis Note Added: 0014514
2020-07-10 15:40 zhb Status closed => feedback
2020-07-10 15:40 zhb Resolution won't fix => reopened
2020-07-10 15:40 zhb Note Added: 0014517
2020-07-10 15:54 zhb Note Added: 0014518
2020-07-10 15:54 zhb Status feedback => assigned
2020-07-13 12:32 the_nic Note Added: 0014521
2020-07-13 15:37 zhb Note Added: 0014522
2020-07-13 15:53 the_nic Note Added: 0014523
2020-07-13 16:04 zhb Note Added: 0014525
2020-07-25 05:56 zhb Note Added: 0014583
2020-07-25 10:27 the_nic Note Added: 0014584
2020-07-25 10:52 zhb Note Added: 0014585
2020-07-25 11:32 the_nic Note Added: 0014586
2020-07-25 14:35 zhb Note Added: 0014587
2020-08-02 13:05 the_nic Note Added: 0014606
2020-08-02 14:04 zhb Note Added: 0014607
2020-08-02 14:06 zhb Note Added: 0014608
2020-08-02 17:07 the_nic Note Added: 0014609
2020-08-03 14:34 zhb Note Added: 0014611
2020-08-03 15:24 the_nic Note Added: 0014612
2020-08-03 15:41 zhb Note Added: 0014613
2020-08-03 15:43 the_nic Note Added: 0014614
2020-08-05 19:22 francis Status assigned => resolved
2020-08-05 19:22 francis Resolution reopened => fixed
2020-08-05 19:22 francis Fixed in Version => 5.0.0
2020-08-05 19:22 francis Note Added: 0014640
2020-08-11 08:43 zhb Status resolved => feedback
2020-08-11 08:43 zhb Resolution fixed => reopened
2020-08-11 08:43 zhb Note Added: 0014663
2020-08-11 08:52 the_nic Note Added: 0014664
2020-08-11 11:28 zhb Note Added: 0014666
2020-08-11 11:28 zhb Status feedback => assigned
2020-08-11 17:49 the_nic Note Added: 0014668
2020-08-12 01:01 zhb Note Added: 0014681
2020-08-12 06:44 the_nic Note Added: 0014682
2020-08-12 08:37 zhb Note Added: 0014690
2020-08-12 13:11 francis Status assigned => resolved
2020-08-19 06:19 zhb Status resolved => feedback
2020-08-19 06:19 zhb Note Added: 0014704
2020-08-19 06:21 zhb Note Added: 0014705
2020-08-19 06:21 zhb Status feedback => assigned
2020-08-19 07:05 zhb Note Added: 0014708
2020-08-19 07:08 the_nic Note Added: 0014709
2020-08-19 07:20 zhb Note Added: 0014711
2020-08-19 10:24 ludovic Status assigned => closed
2020-08-19 11:58 francis Status closed => resolved
2020-08-19 11:58 francis Resolution reopened => fixed