ITUGLIB: News from ITUGLIB on our Procedures

Some of you have asked us, how does ITUGLIB run its system with all that Open Source kicking around. With a great deal of testing and controls, actually! There are some moving parts involved in this that might interest NonStop Insider readers.

Git, of Course

Some of our contributors and sponsors already are involved in the Git project. ITUGLIB has leveraged that tool to turn it into a change monitor. Git is really good at detecting changes in the state of a folder in OSS, and paired with NSGit can monitor GUARDIAN subvolumes for ENSCRIBE modifications. We have set up a repository to monitor all of our configuration files on $SYSTEM. We are also adding repositories to monitor directories in OSS, like /etc.

Checking whether someone or something modified a configuration file is as simple as running:

git status

using cron or NetBatch, or on demand through TACL or OSS, which better come up with nothing. If we get a hit that something was modified, it gets investigated immediately. Git uses pretty fast SHA1 hash signature calculations to detect changes, so even someone being deliberately bad and resetting the timestamp will not get by our Git monitor. This lets us run the system with a fair amount of confidence that things are not changing accidentally or by something in the Open Source area – it’s rather like a rudimentary virus detector for NonStop. For operational/executable files that are not root/SUPER.SUPER protected, we can include these in the scan.

Jenkins, for Testing

Jenkins has become one of the most interesting test coordinators – or continuous integration tools – out there and, even better, it now runs on NonStop J and L series! Jenkins watches our own Git repositories, as well as the Open Source ones we pull from places like OpenSSL, and triggers actions when changes in the repositories are detected. These actions can range from pulling the new changes down from the remote sites, running the compiles, and most importantly running the tests. Some of these tests are really long running, like the full Git suite can sometimes take 36 hours to run and no one really wants to sit and watch the test cases crawl by. Jenkins captures the output for us so we can review the results later if there is a problem, or triggers a packaging job if the tests work.

We have split our configuration into two Jenkins instances, one to do builds, tests, and packaging, and the other, under a different user id, to do installation and distribution to the website. The install/distribution Jenkins has elevated security. We don’t let the build Jenkins get anywhere near root or SUPER.SUPER or even installing software. Jobs in the install Jenkins are initiated by real humans who have reviewed the test results and are satisfied that things are all happy before a product goes live or gets published on our website.

New Things Coming

Based on some feedback from our users, ITUGLIB now is protected via our own brand new SSL certificate – actually it is Connect’s. This allows you to have the confidence to know that the software you download is actually from us and not from a hacker sitting in the middle somewhere.

In the near future, we are also going to add SHA1 hashes to each package that you can verify for integrity. One of our members, Randall Becker (the author of this article), has donated his sha1tool which is now public domain to allow you to run the validation on your own system. We will be publishing that tool on the website as we roll this change out to you. A really important thing you should all know, however: The source code for Open Source packages often has to be slightly modified to run on NonStop – even if it is just the Makefile or Configure scripts. This will cause the original hash from the Open Source website to be different than what we publish. A lot of effort sometimes goes into the porting of Open Source applications to NonStop (even simple things like root is -1 on NonStop and 0 on Linux), and this hash difference is a necessary side effect.

In Summary

The next time you are looking for little somethings to help with one of your NonStop projects, make your first stop at Connect’s ITUGLIB. You can save yourself the porting work that’s already been done!

Bill Honaker, ITUGLIB Engineering Team, XID Software, Inc., Euless, Texas, USA, http://www.xid.com

Randall Becker, ITUGLIB Engineering Team, Nexbridge Inc., Richmond Hill, Ontario, Canada, http://www.nexbridge.com