In late 1974 [1] I was working in the London Midland region signalling drawing office in Carlow Street, just round the corner from Mornington Crescent. British Rail used to circulate a weekly list of vacancies, and one week I saw a requirement for a junior system programmer for the new TOPS project for the wagon fleet management listed. As it said "no previous programming experience required" I applied without a clue of what a junior system programmer actually was or did.
After few weeks I was called for an interview, and had to do a pre-interview aptitude test in a room with about 30 other people. The test was all about numbers in numbered boxes. After about half the allotted time I had finished, and checked it over twice and was getting restless, when I noticed there was only one other person in the same state, whilst the person next me didn't seem to be even half way through. So the two of us who had finished got up at the same time and handed in our papers. Interestingly enough we were the only two who were called back for a full interview and were both subsequently offered jobs.
I was one of the last people to join the project, which had originally received approval in late 1971. The original staffing for the systems programming side of the house had drawn on the best available talent at other pre-existing BR computer centres at Swindon, Peterborough etc. but as none of these centres used IBM mainframes a number of non-BR people who were well versed in IBM software were also recruited. For the application programming teams people were likewise drawn from the existing computer centres, and also a number of railway operations people were trained up as programmers, so that the teams had a good cross section of experience. In addition a number of people from the general BR annual graduate intake were recruited.
The TOPS project HQ was based in Blandford House, in Melbury Terrace, London. It was a nondescript building next to Marylebone station which had previously been occupied by part of the electrical engineering research department who had moved to the new research laboratory in Derby. The building had no external indication of purpose apart from a brass plate with just the address on it, and blacked out windows on the ground floor. The entrance to building was through solid wooden doors with an airlock like arrangement, and I was given a security card with associated pin number.
The communications equipment was on the ground floor, and the computer machine room occupied the whole of the first floor, which because of the shape of the building was a long rectangular space that looked very impressive when standing at one end and looking down the total length.
The second floor housed the management office and support services. All the programming staff were on the top floor which was one big open plan space with thick deep pile carpet of the same design as used in first class carriages of the time. In reality it was a converted attic with glass down one side which was very light and airy, but large area of glass was not good news in the long hot summer of 76. The building came to be known as the Blandford House Computer Centre (BHCC)
When I joined, the project had been under way for quite some time, with all the hardware up and running, along with the TOPS software, which had been acquired from Southern Pacific in the USA, already rolled out, and the computer centre was fully staffed.
The hardware consisted of
2 X 168 OS/370 mainframe computers with
4 Mb of RAM storage each, and several banks of 3330 disk drives which used removable disk packs.
One mainframe was dedicated to running TOPS using the MFT operating system, and this was configured with three partitions:
The software behind MCJ/MEJ/MPJ was written in IBM Assembler (IBM Basic Assembly Language or BAL) for reasons of speed and size. Each job consisted of a number of tasks, so one of the tasks within MPJ was FRM (File request manager) and that handled all the file requests as TOPS used its own file structure with direct access, index and sequential files rather than a proprietary DBMS. The wagon file was a direct access files with a system of buckets, baths and reservoir in which each wagon had a record of around 320 bytes with a 10 byte key[3]. Another task was the appropriately named TOPSCOP. It's roles included detecting any transactions that had got into a CPU loop and aborting them. It also detected when two transactions got into a deadly embrace where one task held a resource the other was waiting on and vice versa. When this happened it would apply some rules and abort one of the transactions.
The application programming was done in TOPSTRAN which was a set of assembler macros that that hide the complexity of the underlying assembler code from the application programming team. There was a batch test program that simulated the online system, so that first cut testing did not an online test environment. The batch program was accompanied by a comprehensive set of test data, and it was setup so that the data was, by default, restored back after a batch test had finished, making it very easy and quick to test changes, against a predictable set of test data.
Meanwhile the offline machine used MVT/ASP and on a day to day basis it ran development and testing work, but it's prime role was to be available as a backup machine when the online machine had a hardware failure or needed to be taken down for maintenance. It should be remembered that TOPS was a "real-time, online" system to use the jargon of the day, but in modern parlance it would be called a 24 x7 (even including Xmas day) operation [8], and TOPS was probably one of only half a dozen such systems in the UK in the 70's. Unlike most mainframes at the time the online machine did not stop processing for overnight backups, and had a system of backups, log files, journals etc being spun off after midnight.
Various references on the web say TOPS was implemented in 1973 but I have had an email from one of the people who joined the project right at the outset in '71 confirming that TOPS rollout started in '73 and it was fully implemented by October 1974. I think the 1973 date refers to the new locomotive numbering scheme which had been introduced to ensure unique numbering for every loco and wagon, and this page seem to indicate that the loco renumbering was only completed in 1975 as it states "the TOPS renumbering took a little time to gather speed but was completed by the latter part of 1975" . However TOPS was not the reason for renumbering as that was the result of a BRB CM&EE (British Railway Board Chief Mechanical and Electrical Engineer) initiative of the early 1970s known as the Unique Numbering Scheme, which had a great deal to do with their introduction of a mainframe system the acronym of which was (and is) RAVERS.
Each depot was originally mean to have a IBM card reader/punch but early trials found that these could not cope with the dusty conditions of the freight yards (think next door to coal mine for some of them) so instead each one had a mini-computer (Datapoint 2200 [6]) emulating the IBM card reader/punch. These still had cards but these were 120 column cards that were much smaller and were less sensitive to dusty conditions. The emulation was considered sufficiently novel by the Open University that it was featured in one of their computing modules. After about a year or so the minis were reprogrammed to use a system called "cardless TOPS" which effectively replaced the physical cards with virtual cards. This was a novel concept at the time and became the subject of an Open university computing module.
My first job was to produce a performance statistics reporting system written in assembler and PL/1. The assembler routine, which included a channel program, was used to access the data stored on a dedicated 3330 disk pack with one cylinder per days data (a 3330 disk pack held 404 cylinders, conveniently just a bit bigger than the 366 required for a leap year). No statistics package such as SAS existed at that time, so reports, histograms etc had to be hand crafted in PL/1.
Some time after that I wrote another reporting tool. The person in charge of BHCC had visited his equivalent in the states and came back with the source code for newly written transaction called "AB". This was a general purpose flexible query tool. It was soon implemented and proved very useful. After about six months or so the system performance reports showed a real peak about 06.00 and my manager then discovered that this was due to a small number of AB transactions that these were being run as time dependant transaction early in the morning causing a disproportionate system load. The AB transaction used a log file of it's own to provide a copy of the input query text. It then fell to me to write a report that married up this logfile with existing logfile that provided details of the files, and I/O count, for each transaction run by the system. Then my report was to provide a summary of the highest usage AB transactions and sort them in various ways. I was sceptical whether this would help us, but it turned out my manager was spot with his plan. AB transactions, once written, could be shared to other users, and then these users could edit the query text to suit their purposes. This, in it's self was not bad. My report highlighted that one particular transaction had got passed around so much that eventually the query text generated resulted in code was scanning a whole file to find one value and not doing a direct read. The cure for this was to write a brand new efficient transaction that could replace the use of the AB transaction for the particular business need. Decades late I found myself in a software products company supporting FQS (Friendly Query System), and lo and behold somone complained about poor performance and the reason was a query shared between users and the original efficient query turned into a poorly performing because of reading too much data due to an altered shared version . History does repeat itself ... well, sort of.
By the mid-seventies the BR wagon fleet probably numbered some 400,000 wagons, and prior to TOPS was still being managed manually, so it is probable that no-one knew the exact size of the total pre-TOPS wagon fleet because of the large number of wagons that pre-dated nationalisation.
With a big project like TOPS a phased implementation made a lot of sense for the following reasons
The way in which TOPS went live was quite unusual as it was done geographically, starting in Penzance area in 1973 and gradually rolling out slowly across the country with the tip of Scotland being the last to be live online, with the entire country live by October 1974 [4 ]. There was an invisible virtual line delimiting the TOPS enabled part of the UK from the non-TOPS rest of the country, and as wagons crossed over this line, those that were not already on TOPS were added to the system every time the operations staff came across them in freight trains. Additionally moving up the country in front of this invisible line were the training staff who trained people in the next area to be cutover so that the TOPS training could be spread out over time with the minimum number of trainers.
As the rollout of TOPS progressed, by the time it was approaching Birmingham, the technical planning manager, who was responsible for all the hardware and software, had done some extrapolations of system load, and it looked like a single 168 mainframe would not be able to cope with the workload once the whole country was cutover.
To ensure that adequate processing capacity was available an additional IBM 158 mainframe was procured and installed. The plan was that TOPS software would be updated so that the 158 would run the MCJ program and a 168 would run MEJ/MPJ, or in the event of the 158 failing the other 168 would cover for it.
No sooner was the 158 up and running and being tested, than the processor load started to stabilise. It became apparent that as each new area came onto the system, its transaction load was initially higher than anticipated because users would try things out by running additional enquiries etc, and also make more mistakes, and hence resubmit transactions ,so these two factors artificially increased the number of transactions per day. However over time the processor load then fell back to around about the original estimated workload as once users were comfortable with the system they made fewer mistakes and spent less time just trying things out.
Also when TOPS was being used in the USA it covered a number of times zones, but in the UK it was all in one time zone, so this compressed the morning and evening peak busy periods in to much shorter time windows. In the 70's freight was mainly run overnight, often using the same electric or diesel locos that hauled mainline passenger trains during the day time with freight trains leaving between 23.00 and midnight and arriving around 06.00, so these two time slots were the peak busy period for TOPS activity.
Finally the system programming team for TOPS had identified, one by one, a number of improvements that could be made to the software to eliminate bottlenecks etc., and as each of those improvements was implemented the software became that little bit more efficient.
By the time the whole country had been cut over in November 1975 and the system had settled down it was handling some 200K transactions a day.
Things did not stand still after implementation, and the system itself had additional tuning to improve performance and various add-ons developed.
The data that was spun off overnight was put to good use, and one application was an offline historical database using IBM's IMS dbms and this was appropriately called TARDIS (TOPS ancillary retrospective database information system). Initially requests for information were done with paper forms, but that system was replaced by a new TOPS transaction which submitted a request for overnight processing overnight with the report being printed and sent out in the internal post, but eventually the output was routed back via TOPS to the person who had originally submitted it.
In the late seventies a loco database, also using IMS/DB, was developed. The system requirements for this had to meet the needs of both the electric and diesel engineering departments, which could at times be quite different e.g. electric locos had a time based maintenance intervals and diesels were mileage based.
TOPS was a major culture change for railway operations, and there was initial quite a lot of scepticism, and when the system went down due the hardware failure the field staff referred to the manual paper system used in that situation as BOTTOMS (back on to the old manual system). TOPS was meant to ensure better utilisation of the wagon fleet so that departments such as Civil Engineering were meant to "pre-order" wagons for use on weekend engineering work, and it took a while for them to trust the system to schedule the ordered wagons to turn up on time rather than hoarding wagons during the week ready for the weekend.
The figure I heard quoted was that TOPS shrunk the wagon fleet down from 400,000 down to 200,000 wagons within two years of implementation with knock savings in maintenance costs and freeing up of space occupied by sidings for other use.
Round about 1977/78 the second 168 operating system and scheduler/spooler was upgraded from MVT/ASP to MVS/JES2 to take advantage of the much improved reliability and throughput that MVS offered. Meanwhile the research department at Derby had an IBM mainframe (a 145 ?) running VS/1 and this had a very cpu bound workload, which nicely complemented the I/O (input /output) bound workload of the Marylebone offline 168. So the Derby mainframe was decommissioned and the Derby user were given remote access to the MVS machine via two links for time sharing (28k bandwidth) and a RJE(Remote job entry) terminal and printer. In London an internal bureau service was offered on this mainframe to departments such as Operational Research, Strategic planning Civil engineering etc.
Also during this period the offline machine started running non-TOPS related tasks such as Carousel, a merry go round train fleet scheduling program which was a joint venture between BR, CEGB and NCB with input from the Met Office. This was used to help predict the coal and electricity demand for the coming week based on the likely weather, and achieve the required coal delivery to power stations with the minimum of train movements. As the TOPS programming activity dropped back the BHCC programming teams, and people were used on other activities such as helping to create a reservation system(STARS) used initially on the East Coast mainline, which was based on an ACP/TFP system acquired from "Société Nationale Maritime Corse Méditerranée" (SNCM)[5] who operated from France to Corsica, Tunisia and Morocco. SNCM had built their system to be compatible with airlines and used very similar user messages
TOPS was still updated from time to time which included things like combining MCJ and MEJ, and replacing TCAM with a CICS interface for input.
The evolution of IBM hardware also affected TOPS as disk space moved from the original 3330 removeable disk packs to 3350 hard drives and then on to 3380,3390. Changes for the latter were an issue to start with as 3390 track length exceeded 32k.[7]
It was a bit of an irony that changing to a newer disk type achieved no disk space savings as the disks themselves were just more empty.
At some point in late 80's/early 90's TOPS was transferred from BHCC to the Crewe Computer bureau.
After the move to Crewe a hot standby facility was created by writing all the file changes at Crewe to logfiles. When the logfiles were spun off they then got transferred and applied to the database at the Nottingham computer bureau. The system was tested but never used in anger.
Blandford House was demolished after TOPS had been moved and the site is now occupied by the BNP Paribas building which was built in 1997.
At privatization, ownership of TOPS was passed to Railtrack, but BR Computing, also privatised, ran it.
Eventually, with rapidly shrinking mainframe usage, it was deemed uneconomic to maintain data centres at Crewe, and Nottingham. Thus the TOPS system was transferred yet again. This time to an ATOS data centre on the south coast and installed on a spare partition on a large, up to date ATOS mainframe[2] .
After TOPS had been running for a couple of years we had about three or four system crashes, over about a four month period. I sat opposite the systems programmer responsible for diagnosing memory dumps from system crashes and he was very perplexed by these, which was unusual for him as he normally got to the bottom of things quite quickly. It almost got to be a regular Friday afternoon thing when he would haul one of these memory dumps (produced on paper in those days) onto his desk and would pore over the hexadecimal contents through a nicotine haze and mutter "this should not be happening". Then one day the TOPS mainframe went down with a hardware failure which we were told was due to a tiny, slow water leak over the CPU card. My colleague immediately threw out his three foot high paper archive of memory(core) dumps because his conclusion was that the water leak was causing oddball processing side effects in the CPU circuits and he was spot on as we never got any more of these errors.
Around the same time the application programming team had a software error reported where a train had en route between depots had one wagon mysteriously disappear but a totally different one had been added to the train. The operations staff out in the field were convinced it was a software error as in the 70's computers were still viewed with deep suspicion. Eventually the penny dropped with someone in the application programming team who asked "Can you check the wagon has the same number on both side" which, of course, it hadn't. It must have been trundling round the railway networks in that state for weeks if not months.
The TOPS system, as implemented, hide explosives as a cargo code from general view. This was particularly important as the IRA were in the middle of a major UK bombing campaign at the time. Ironically the Balcombe Street siege in late 1975 was just round the corner from Blandford House, although the IRA probably didn't know that. A few years later the facility to cloak explosive cargos was expanded to cover valuable cargos such as whisky following concerns that TOPS could be used to help criminals specifically target these cargos en route. Probably one of the earliest examples of concerns about crime and computers.
Les Smith
December 2009
(updated : April 2010, January 2011, April 2011, November 2011, November 2019, October 2020, May 2022)
British Transport Films made a number of documentaries on TOPS as below, but none have yet made their way onto the DVDS released by BTF
1.
I originally wrote "In March 1975" but a couple of months after I first put this page up I actually got an email from the other person from the aptitude tests who was successful who I haven't seen since the early eighties. He corrected me to says it was late 1974 and on reflection I have a vague recollection that my signalling section leader was slow to release me to move across. A bit ironic as I was working on signalling for the Manchester Picc-Vic scheme and the project got cancelled two years later, although it did eventually see the light of day as it evolved into part of the Manchester metrolink light railway.
2.
My ex colleague mentioned in note 1 left the TOPS project a little after me, and he says he worked for Abbey Life at their Data Centres near the south coast. Then through a series of takeovers and acquisitions he has been moved via Unisys, and then Capita to one of the large computer services companies, and he says "What is bizarre is that my current company also run the railway mainframe systems including TOPS at their Data Centre in Hampshire."
3.
My thanks to Mike Kerford Byrnes for reminding me about buckets, baths and reservoirs.
4.
An email from Brian Bushell, who had joined the original TOPS team in October 1971 and responsible initially for MPJ and then technical of commissioning/rollout, gave me a lot of details about the project before I joined and also the fact that my memory was playing tricks with dates when, in my original version of this page, I said said that rollout started in '75 . He confirmed the rollout dates by saying "But I'm afraid some of your dates are a bit amiss! Devon and Cornwall did indeed go live in 1973, and the entire country was live by October 1974. Project Manager Bob Arnott invited us all up to a celebration buffet on the top floor of Blandford House."
Thanks to Bob Parsons for confirming the Corsician ferry connection and additional info on this.
Thanks to Peter Ward for confirming the mini computers running the emulation software were Data point. He wrote to me saying
"I was a systems engineer at the time for Datapoint based on the North Circular Road and I did install and repair some 2200 machines in BR locations. Interestingly one particular location kept losing its data intermittently and was causing quite a problem. After several visits I noticed the error occurred when a class 87 locomotive, which was parking in a siding next to the small building in which the machine was located, was brought into service by the driver and when he raised the pantograph the moment it touch the overhead supply a very large magnetic gauss field was generated which wiped the stored magnetic data. Enclosing the machine in a faraday cage cured the problem. Computer engineers don't get to do that nowadays"
7.
My thanks to Peter Beever for providing information on updates to TOPS in the eighties/nineties and also the transfer of the system to Crewe, and then to the ATOS site in Hampshire.
8.
In theory TOPS was online and available 365 x 24 i.e, all day, all year round, but the service needed to be deliberately stopped twice a year to allow for the clocks going forward or back. Going forward was easy, the system was stopped and the time changed. This probably probably needed an IPL(Initial Program Load), the mainframe equivalent to a PC reboot. When the clocks went back the message queue was simply stopped for an hour and allowed to start again after an hour had elapsed. No doubt a more elegant solution was found as the years rolled by.