ITUGLIB: A uniquely valuable resource for your modern NonStop (Part 2)

Git and ITUGLIB Automation

Git is one of those Open Source tools that has taken on a life of its own. Starting as a replacement for the ubiquitous version control system CVS, the goal of the git project was to fundamentally change how we manage code to allow projects of the size and scale of LINUX (or BASE24 for that matter). Because git had to support potentially thousands of developers all working away on our own thing, trying out our own ideas, there had to be some way of allowing us to work at any time instead of having one central place where the code resides. The requirement of being constantly connected in order to work had to be thrown out. How many of you have written code on a non-WiFi airplane and were able to commit the code and switch branches to work on something else within the same product? Because of git, a few of the ITUGLIB team members have done that. To support this not so off the wall idea, git built software distribution and transport right into its core from the beginning. SSH and SSL are automatically launched as layers within commands, when you are working with a repository that is not local, while pushing code up to a main repository, for example. This idea fits so well into what ITUGLIB is doing that we just could not resist taking advantage of it.

As an example, we are linked up to the official OpenSSH repository, where we pull their changes on a regular basis. These changes are merged, by a human who looks at the merge results to make sure things are still sane. Once we decide that the changes look good, they get merged into our ituglib-release branch. Then the fun begins. Within moments, Jenkins, our build/test automation too (which is also Open Source) sees the change in git and pulls the change into its own OpenSSH repository clone, and starts a build job. If that build works, the test job is automatically launched and the OpenSSH sanity tests run, sometimes for hours. And even better for us, if the tests pass, Jenkins then launches a packaging job that stages the whole OpenSSH code for distribution on our web-site – all while we are happily sleeping.

The history stored in git, through all these operations, is very interesting. It contains the GPG signed tags from the OpenSSH contributors, so we know who made the actual fixes to a core security product. It also contains the footprints of whichever team members were involved in the merge. Jenkins tells us who initiated the final publish step that moves the code to the website. And all of this is authenticated using SSH so it is very clear who did what, all the way along the process. We are in the process of making these repositories available for you to pull down our changes so the footprints will show up all the way to your own production environments – including any changes you might have had to make along the way.

The Plan for Contributions

The software transport built into git does not run in only one direction, unlike most other VCS systems. In git, assuming you have the permissions in place, you can push or pull code in any direction. For ITUGLIB users, this would mean pulling code down to your environment; but for contributors, you would be able to push your changes up to ITUGLIB when you are ready to contribute, fix a bug, or contribute a new feature to an existing package. This could happen from within ECLIPSE, or from your NonStop, assuming your company permits it, of course. Our process model for contributions is being designed around git’s Fork Flow model, where you would have a working copy of a repository secured specially for you at ITUGLIB. The plan is for you to use an SSH key pair to push your code from your local repository clone to your dedicated repository at ITUGLIB, and then tell us when you are ready for your change to be pulled into the official repository. You could theoretically do that today if your contribution is ready for us at GitHub or Bitbucket.

NSGit, When Your Contribution is a GUARDIAN Package

A new product, NSGit, has recently come onto market that addresses one of the keys gaps associated with git; specifically, git is an OSS product that only knows about unstructured files. NSGit extends git by providing support for ENSCRIBE artifacts. Running on the GUARDIAN side, NSGit provides a git-like user experience for GUARDIAN files, including data files, DDL dictionaries, and POBJ SCOBOL components. Users can do all the cool things that git with GUARDIAN packages. This makes it ideal for building release packages that can move through your software backbone through products like GitHub Enterprise from development to production, managing your development branches for parallel projects, and keeping track of operation’s system configuration files

Nexbridge has made its NSGit product available to ITUGLIB for managing GUARDIAN packages. A few of these have been put into git using NSGit, including CMON, SWFSYS, and OFFENDER. We are still working out details for contributors, but the contribution method would be virtually the same as for OSS contributions. NSGit helps the team by translating a GUARDIAN package, with code 101 and other ENSCRIBE artifacts, into git with metadata that travels along with the git repository and commits so that packages can be reconstituted back into GUARDIAN by users. Our plan as a team is to put GUARDIAN packages into git as needed, based on the frequency of contribution, but you will be able to contribute GUARDIAN as well as OSS using the same Fork Flow process.

Summary

The NonStop system infrastructure has continued to change. It started as a proprietary CISC processor using legacy languages such as TAL and COBOL. It has migrated to RISC chips, then to Intel Itanium processors. It’s now a modern, open system running on x86 processors, and even runs in a Virtual Machine. The developer’s environments have also modernized to include Eclipse and git. And they all still support the legacy languages, since a lot of the ‘business logic’ written in them is still valid.

In much the same way, ITUGLIB has been changing over the years. Today, it supports the use of a myriad of Open Source programming knowledge, and is about to change its delivery mechanism into one familiar to every Developer straight out of college, and most any developer that has touched an Open system such as Linux or Unix. With these tools at hand, the riches of Open Source are available to the developers of NonStop solutions, and they get the ‘nines’ and scalability for free. Who can argue with that?

Biography callout:

Bill Honaker is a consultant with XID, Software, Inc., serving the company as its Chief Technology Officer. He has over 37 years of NonStop experience as a Solution Architect, Performance Analyst, Systems Integrator, System Manager, Database Designer, and anything else our customers will pay us for. XID has been keeping its customers satisfied since 1989 by treating them as we would want to be treated.

Bill has been a volunteer with both ITUG and Connect since 1988, which partially explains his gray hair. In the past, he served as both editor and director for the Connection magazine, Treasurer in 1993 and 1994, Vice President in 1995, and as ITUG President in 1996. He currently serves as the President of the N2TUG Chapter of Connect, the moderator of the NonStop Open SIG, and the Lead of the ITUGLIB Engineering team. Bill has cared for the ITUGLIB for ITUG and Connect at the XID offices since 1990, in the days when copies were delivered to members on a big reel of magnetic tape.

Bill Honaker, ITUGLIB Engineering Team, XID Software, Inc., Euless, Texas, USA, http://www.xid.com

Randall Becker is a process and SCM guru with Nexbridge Inc. He has been on the NonStop since 1983 in various capacities including solution design, performance, SQL/MX migrations, and developer of a number of version control products.

Randall has also been a volunteer with ITUG and Connect since 1989, and was director of CTUG for over a decade. He has been instrumental in designing the automation and management processes for ITUGLIB and in defining the contribution procedures using git. Randall has also written for and spoken at various ITUG and Connect events over the years as well as for the Disaster Management community.

Randall Becker, ITUGLIB Engineering Team, Nexbridge Inc., Richmond Hill, Ontario, Canada, http://www.nexbridge.com