OpenSSL and “DLL Hell” The Elephants in the Room

It was 11:30pm last week and an email came into the support system: a customer had gone down. They were in a panic because “Nothing was working” – well, not nothing, but just enough to get everyone out of bed. A new RVU had been installed and now applications would not run. This was a first for this customer, who had never seen this kind of outage before.

Yet for those of us who have been living in “DLL Hell” for years, if not decades, knew this was coming. In fact, we had warned this customer months in advance of this potential, and we were prepared (fortunately).

Just delete the DLLs in /usr/lib associated with SSL and you will be fine, I replied. Yes, I said more than that, but that is the gist of it.

And they were, in moments. Applications started and everyone was up and running again.

So, what really happened? To be mostly honest, I was writing a completely different article at the time, trying to fall asleep. Or trying to write the article and not falling asleep. But that’s not important right now. What is important is the managing DLLs in OSS is really easy but really difficult at the same time – the “Hell” part.

What is OpenSSL

OpenSSL is a security product that is the basis of many crucial components that run in OSS. Some infrastructure applications, like IBM MQ, git, rsync, curl, and OpenSSH use OpenSSL for cryptography and as part of key exchanges. It is also the basis of many commercial products. You may be using OpenSSL without even realizing it.

Understanding DLLs in OSS

Dynamic Link Libraries (DLLs) are useful things, where code that is shared between different programs can be put into libraries that are only loaded into memory once. This can greatly improve performance on systems with smaller amounts of physical memory. But far more importantly, DLLs allow you to update critical components when fixes are available without having to reinstall your entire application. This is the case with OpenSSL. Patches happen often and usually soon after some hacker somewhere out there in the dangerous world finds a vulnerability. ITUGLIB gets the patch, usually within hours, and the build system churns out a new version of the DLL, often within the time it takes to build and run the regression suite. A customer can then drop a new OpenSSL package onto their system without a SYSGEN and sometimes without even taking the application down – if care is taken.

DLLs are found by OSS using the file name of the DLL. The list of directories where OSS looks is specified in the environment variable _RLD_LIB_PATH. Once a matching name, like libssl.so is found, that’s the one that gets used. Full stop. Big Period. End of Paragraph.

The Elephant

Up until about May 2020, there was only one visible version of OpenSSL residing in OSS. At least only one that you needed to worry about, and that was version 1.0.2 in /usr/local/lib.

As of today, there are a bunch of OpenSSL versions:

• 0.2u and 1.1.1g from ITUGLIB
• PUT and SPT threaded versions of 1.1.1g from ITUGLIB
• IBM MQ has their own version
• HPE T2813 has a 1.1.1 version
• 0.0 is under test by ITUGLIB
• There are more

If you have the wrong version in your _RLD_LIB_PATH, your program will not work. And that is exactly what happened in the support case. T2813 was introduced as a mandatory component in L20.05 and it broke the customer’s application because of an incompatibility.

The Bigger Elephant

The different versions of OpenSSL are not binary compatible with each other. So, you cannot just use whatever version is on your system. You must know and use exactly the version you need… or else. The threading models – POSIX User Threads (PUT) or Standard POSIX Threads (SPT) – and memory models also make the DLLs incompatible, so you also must know that too. And all of these get resolved with the same library file names. It is a very heavy elephant indeed.

The Easy Answer

Most of the time, getting rid of T2813 seems to be the most expedient way to solve the situation. You should watch my webinar at https://www.comforte.com/webinar-managing-openssl/ and work out a strategy that works for you. Or you can contact me to help out – yes, a shameless plug.

The Long Term

Eventually, hopefully, and maybe sooner than later, the OpenSSL codebase will include support for NonStop right out of the box. That means that anyone should be able to build the version they want with the threading and memory models required for your application. One day. Then I might get some good sleep for a change