Infinite Loop in OpenSSL s_server

Bug #1947588 reported by Matt Caswell
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
openssl (Ubuntu)
Fix Released
Medium
Unassigned
Focal
Fix Released
Medium
Unassigned
Impish
Fix Released
Medium
Unassigned
Jammy
Fix Released
Medium
Unassigned

Bug Description

[Impact]

The TLS test server `openssl s_server` can very easily be led into an
infinite loop if configured with incompatible settings and used via
DTLS. This makes it harder to test one's TLS configuration.

[Test plan]

In one session:
$ openssl s_server -nocert -psk 01020304 -dtls1
In parallel:
$ openssl s_client -dtls1 -psk 01020304

The server session will enter an infinite loop:
Using default temp DH parameters
ACCEPT
ERROR
140247926990208:error:141FC044:SSL routines:tls_setup_handshake:internal error:../ssl/statem/statem_lib.c:109:
ERROR
140247926990208:error:141FC044:SSL routines:tls_setup_handshake:internal error:../ssl/statem/statem_lib.c:109:
ERROR
... etc...

[Where problems could occur]

The patch is fairly self-contained, so regressions should only occur in
the `openssl s_server` application, and not in the libssl or libcrypto
libraries.
However, the patch could break said server, which might be used in e.g.
autopkgtests.

[Original report]
Launching openssl s_server as follows:

$ openssl s_server -nocert -psk 01020304 -dtls1

And using openssl s_client to connect to it like this:

$ openssl s_client -dtls1 -psk 01020304

Results in s_server entering an infinite loop:

Using default temp DH parameters
ACCEPT
ERROR
140247926990208:error:141FC044:SSL routines:tls_setup_handshake:internal error:../ssl/statem/statem_lib.c:109:
ERROR
140247926990208:error:141FC044:SSL routines:tls_setup_handshake:internal error:../ssl/statem/statem_lib.c:109:
ERROR

...and so on...

I have confirmed that upstream OpenSSL does not have this issue in a default build of 1.1.1j or 1.1.1k. Upstream 1.1.1l has a different bug with these commands (https://github.com/openssl/openssl/issues/16707) and it was while working on the fix for that issue (https://github.com/openssl/openssl/pull/16838) that I noticed this problem in the Ubuntu packages.

$ lsb_release -rd
Description: Ubuntu 21.04
Release: 21.04

$ apt-cache policy openssl
openssl:
  Installed: 1.1.1j-1ubuntu3.5
  Candidate: 1.1.1j-1ubuntu3.5
  Version table:
 *** 1.1.1j-1ubuntu3.5 500
        500 http://gb.archive.ubuntu.com/ubuntu hirsute-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu hirsute-security/main amd64 Packages
        100 /var/lib/dpkg/status
     1.1.1j-1ubuntu3 500
        500 http://gb.archive.ubuntu.com/ubuntu hirsute/main amd64 Packages

$ openssl version -a
OpenSSL 1.1.1j 16 Feb 2021
built on: Mon Aug 23 17:02:39 2021 UTC
platform: debian-amd64
options: bn(64,64) rc4(16x,int) des(int) blowfish(ptr)
compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -Wall -Wa,--noexecstack -g -O2 -ffile-prefix-map=/build/openssl-5U8yxE/openssl-1.1.1j=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_TLS_SECURITY_LEVEL=2 -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM -DMD5_ASM -DAESNI_ASM -DVPAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DX25519_ASM -DPOLY1305_ASM -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2
OPENSSLDIR: "/usr/lib/ssl"
ENGINESDIR: "/usr/lib/x86_64-linux-gnu/engines-1.1"
Seeding source: os-specific

Related branches

CVE References

Revision history for this message
Simon Chopin (schopin) wrote :

Hi Matt, thanks for taking the time to report this, as well as investigating the different versions that might be affected.

I just checked, and both Impish and Jammy are affected as well (the latter using OpenSSL 3.0).

Revision history for this message
Simon Chopin (schopin) wrote :

Disregard the (now deleted) comment regarding Focal, I got mixed up in my terminals. Focal is affected, but Bionic is not.

I suspect this is caused by our patch that changes the semantics of security level 2 to also drop support for (D)TLS < 1.2.

tags: added: rls-ff-incoming rls-jj-incoming
Changed in openssl (Ubuntu Focal):
status: New → Confirmed
Changed in openssl (Ubuntu Impish):
status: New → Confirmed
Changed in openssl (Ubuntu Jammy):
status: New → Confirmed
Simon Chopin (schopin)
tags: removed: rls-ff-incoming
Changed in openssl (Ubuntu Jammy):
importance: Undecided → Medium
Changed in openssl (Ubuntu Impish):
importance: Undecided → Medium
Changed in openssl (Ubuntu Focal):
importance: Undecided → Medium
tags: added: fr-2135
tags: removed: rls-jj-incoming
Revision history for this message
Simon Chopin (schopin) wrote :

I had a look at what's going on there. My understanding (with the caveat that the code of s_server is quite hard to follow, even within GDB) is that when the socket receives the packet, the server goes on and try to establish a connection, only to find out that it cannot because it has an inconsistent configuration (DTLS1 being disabled on seclevel 2 on Ubuntu), thus erroring out early, before it actually reads from the socket, thus triggering the loop all over again. This does not happen with TCP-based protocols, I assume because the underlying stream socket is closed (haven't checked the details though).

Fixing this cleanly would probably be a bit tricky (do we want to abort() the application? If not, what do we do with the incoming datagram?) but isn't very urgent either as it is an issue with the s_server code, which AIUI is a debugging tool.

Revision history for this message
Matt Caswell (matt-openssl) wrote :

Thanks for your analysis. Based on your description I was able to find an instance of this bug that impacts an unmodified upstream OpenSSL directly. I've raised an issue for it here:

https://github.com/openssl/openssl/issues/18047

That particular instance only impacts OpenSSL 3.0 - but its the same underlying cause as here.

Revision history for this message
Matt Caswell (matt-openssl) wrote :

FYI, upstream merged a fix for the underlying problem in OpenSSL 3.0:

https://github.com/openssl/openssl/commit/8b63b174b00b0e8c5cefcea12989d90450e04b24

I expect a similar fix to be backported to 1.1.1 soon. Although the specific issue that this bug report is about doesn't impact upstream, I expect that any backported fix will also resolve this bug.

Revision history for this message
Matt Caswell (matt-openssl) wrote :

FYI, upstream have now also merged a fix in the 1.1.1 branch:

https://github.com/openssl/openssl/commit/e04ba889594d84a8805f3d0caeadf0527470e508

If Ubuntu pulls in that patch I expect that this bug should be fixed by it.

Revision history for this message
Simon Chopin (schopin) wrote : Re: [Bug 1947588] Re: Infinite Loop in OpenSSL s_server

Thanks for the follow up! I'll try to fold the fix for this in the next
Jammy SRU, I don't know about other releases yet.

Simon Chopin (schopin)
description: updated
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package openssl - 3.0.3-5ubuntu2

---------------
openssl (3.0.3-5ubuntu2) kinetic; urgency=medium

  * d/p/Set-systemwide-default-settings-for-libssl-users: don't comment out
    the CipherString string to avoid an empty section.

 -- Simon Chopin <email address hidden> Tue, 31 May 2022 13:02:15 +0200

Changed in openssl (Ubuntu):
status: Confirmed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello Matt, or anyone else affected,

Accepted openssl into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/openssl/3.0.2-0ubuntu1.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in openssl (Ubuntu Jammy):
status: Confirmed → Fix Committed
tags: added: verification-needed verification-needed-jammy
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Hello Matt, or anyone else affected,

Accepted openssl into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/openssl/3.0.2-0ubuntu1.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Simon Chopin (schopin) wrote :

On a fresh Jammy LXC container:

root@rational-polliwog:~# dpkg -l openssl | tail -n 1
ii openssl 3.0.2-0ubuntu1.4 amd64 Secure Sockets Layer toolkit - cryptographic utility
root@rational-polliwog:~# openssl s_server -nocert -psk 01020304 -dtls1
Using default temp DH parameters
ACCEPT
ERROR
40472C92B97F0000:error:0A0000BF:SSL routines:tls_setup_handshake:no protocols available:../ssl/statem/statem_lib.c:104:
shutting down SSL
CONNECTION CLOSED
   0 items in the session cache
   0 client connects (SSL_connect())
   0 client renegotiates (SSL_connect())
   0 client connects that finished
   0 server accepts (SSL_accept())
   0 server renegotiates (SSL_accept())
   0 server accepts that finished
   0 session cache hits
   0 session cache misses
   0 session cache timeouts
   0 callback cache hits
   0 cache full overflows (128 allowed)

Marking as verified.

tags: added: verification-done-jammy
removed: verification-needed-jammy
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Matt, or anyone else affected,

Accepted openssl into impish-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/openssl/1.1.1l-1ubuntu1.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-impish to verification-done-impish. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-impish. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in openssl (Ubuntu Impish):
status: Confirmed → Fix Committed
tags: added: verification-needed-impish
Changed in openssl (Ubuntu Focal):
status: Confirmed → Fix Committed
tags: added: verification-needed-focal
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Matt, or anyone else affected,

Accepted openssl into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/openssl/1.1.1f-1ubuntu2.14 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (openssl/1.1.1l-1ubuntu1.4)

All autopkgtests for the newly accepted openssl (1.1.1l-1ubuntu1.4) for impish have finished running.
The following regressions have been reported in tests triggered by the package:

nagios-plugins-contrib/35.20210511ubuntu2 (ppc64el)
nodejs/12.22.5~dfsg-5ubuntu1 (amd64)
swupdate/2020.11-2 (ppc64el)
asterisk/1:16.16.1~dfsg-2 (s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/impish/update_excuses.html#openssl

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (openssl/1.1.1f-1ubuntu2.14)

All autopkgtests for the newly accepted openssl (1.1.1f-1ubuntu2.14) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

trafficserver/8.0.5+ds-3 (ppc64el)
linux-oem-5.14/5.14.0-1042.47 (amd64)
linux-hwe-5.11/5.11.0-61.61 (armhf)
linux-intel-iotg-5.15/5.15.0-1008.11~20.04.1 (amd64)
linux-hwe-5.15/5.15.0-33.34~20.04.1 (armhf)
mysql-8.0/8.0.29-0ubuntu0.20.04.3 (amd64)
puma/3.12.4-1ubuntu2 (arm64)
linux-hwe-5.13/5.13.0-48.54~20.04.1 (armhf)
diaspora-installer/0.7.6.1+debian1 (s390x, arm64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#openssl

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package openssl - 3.0.2-0ubuntu1.4

---------------
openssl (3.0.2-0ubuntu1.4) jammy; urgency=medium

  * d/p/lp1978093/*: renew some expiring test certificates (LP: #1978093)

openssl (3.0.2-0ubuntu1.3) jammy; urgency=medium

  * d/p/lp1974037/*: cherry-pick another patchset to fix regressions with the
    previous lp1974037 one (LP: #1974037)
  * d/p/Set-systemwide-default-settings-for-libssl-users: partially apply it on
    Ubuntu to make it easier for user to change security level (LP: #1972056)
  * d/p/lp1947588.patch: Cherry-picked as our patches make it very easy to
    trigger the underlying bug (LP: #1947588)

 -- Simon Chopin <email address hidden> Thu, 09 Jun 2022 13:20:55 +0200

Changed in openssl (Ubuntu Jammy):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for openssl has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Simon Chopin (schopin) wrote :

Focal verification:

❯ dpkg -l openssl | tail -n 1
ii openssl 1.1.1f-1ubuntu2.14 amd64 Secure Sockets Layer toolkit - cryptographic utility

❯ openssl s_client -dtls1 -psk 01020304
CONNECTED(00000003)
140292984321344:error:141E70BF:SSL routines:tls_construct_client_hello:no protocols available:../ssl/statem/statem_clnt.c:1112:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 15 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
---

Impish verification:

❯ dpkg -l openssl | tail -n 1
ii openssl 1.1.1l-1ubuntu1.4 amd64 Secure Sockets Layer toolkit - cryptographic utility

❯ openssl s_server -nocert -psk 01020304 -dtls1
Using default temp DH parameters
ACCEPT
ERROR
140008339306304:error:141FC044:SSL routines:tls_setup_handshake:internal error:../ssl/statem/statem_lib.c:109:
shutting down SSL
CONNECTION CLOSED
   0 items in the session cache
   0 client connects (SSL_connect())
   0 client renegotiates (SSL_connect())
   0 client connects that finished
   0 server accepts (SSL_accept())
   0 server renegotiates (SSL_accept())
   0 server accepts that finished
   0 session cache hits
   0 session cache misses
   0 session cache timeouts
   0 callback cache hits
   0 cache full overflows (128 allowed)

tags: added: verification-done-focal
removed: verification-needed-focal
tags: added: verification-done verification-done-impish
removed: verification-needed verification-needed-impish
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package openssl - 1.1.1l-1ubuntu1.5

---------------
openssl (1.1.1l-1ubuntu1.5) impish-security; urgency=medium

  * SECURITY UPDATE: c_rehash script allows command injection
    - debian/patches/CVE-2022-1292.patch: switch to upstream patch, and
      apply it before c_rehash-compat.patch.
    - debian/patches/CVE-2022-2068.patch: fix file operations in
      tools/c_rehash.in.
    - debian/patches/c_rehash-compat.patch: updated patch to apply after
      the security updates.
    - CVE-2022-2068

 -- Marc Deslauriers <email address hidden> Wed, 15 Jun 2022 10:38:42 -0400

Changed in openssl (Ubuntu Impish):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package openssl - 1.1.1f-1ubuntu2.15

---------------
openssl (1.1.1f-1ubuntu2.15) focal-security; urgency=medium

  * SECURITY UPDATE: c_rehash script allows command injection
    - debian/patches/CVE-2022-1292.patch: switch to upstream patch, and
      apply it before c_rehash-compat.patch.
    - debian/patches/CVE-2022-2068.patch: fix file operations in
      tools/c_rehash.in.
    - debian/patches/c_rehash-compat.patch: updated patch to apply after
      the security updates.
    - CVE-2022-2068

 -- Marc Deslauriers <email address hidden> Wed, 15 Jun 2022 14:16:37 -0400

Changed in openssl (Ubuntu Focal):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.