How internal competition can often be the most dangerous kind
 |
I’ve written before about my time at Digital Equipment Corporation (DEC) in the late 1990s, working on a disastrous incentive program to build Alpha workstations. This one is another tale of Digital, but for it we need to go back a decade earlier. Back to 1986 when I joined the company, fresh out of college.
I grew up in the late 70s/early 80s, idolizing DEC. It was the cool computer tech company that had produced the PDP-8 mini, which I learned to program on, and later the VAX 11/780, which I had used in some of my earliest computer courses I took while still in high school. Thinking they were the best computer company may have been primacy bias on my part, perhaps. But there was no debating that Digital was a dominant force in the market, holding the number two computer manufacturer spot, behind IBM, and a desirable place for a new engineer to work.
But by the time I was a Senior in college, it was strangely not I that went looking for a job at DEC; it was DEC that found me. I had in fact “uneven” grades let’s say, and I was really just searching to find a job of any sort, as graduation neared. As fate would have it though, a school alumni who worked at Digital came by, looking for a college hire. The professor who’s lab I worked in recommended me, and the next thing you know, I was renting my first car, to drive out to Shrewsbury, Massachusetts for an interview.
I was worried about having to explain some of my grades, but the subject, luckily, never came up. They asked a few digital logic questions which I could definitely handle, and they were very interested in my senior design project, a microprocessor-based LAN/router I had built that later won best project at the school that year. (Having that project to talk about was definitely a lifesaver and welcome distraction from explaining my calculus woes!)
They made me an offer the next day, and there was absolutely no debate in my head about taking it. So I went to work at my dream company, and I still remember the feeling of walking into work on day one and seeing my office and name plate, with brand new VAXcluster-connected VT100 terminal of my own ready for me, and thinking “Holy shit, I’m working for Digital!”
I got an apartment about a mile down the road from the plant, but on day one in ESD, I learned that my assigned project was actually going to be at the office in Littleton, Massachusetts, 25 miles to the north. DEC of the day was this giant, spreading organism with offices sprouting up on an almost weekly basis in one town or another in the western suburbs of Boston. Inter-office hopping was common, and DEC would pay mileage for people who had to travel, and even had a fleet of helicopters to shuttle people around.
Our ESD division in Shrewsbury had the job of designing the memory systems (dynamic ram) for all of DEC’s products. My group worked on the main memory systems for VAX mainframes, which at the time were very complex beasts with a lot of specialized hardware for controlling things like interleaving, error detection and correction, self-test diagnostics, and ram refresh handling.
Later as computers became smaller, a lot of this specialized memory hardware was either not needed, or incorporated into ICs, and memory systems for computers in general became a commodity. By the time of my adventures in the Alpha workstations group in the 90’s, there would no longer be a centralized storage group at DEC, for memory design anyway, and this idea that hardware could become a commodity had stuck in my head and was pushing me to (eventually) move to a software career.
1986 was a different matter. Our ESD group then was busy working on many products across the company, including several very large and as it turns out, competing mainframe designs. I would drive up to Littleton every day to work on my assigned product, which was a new VAX mainframe code-named “Argonaut”. This was to be a third-generation offering from the Midrange Systems Business (MSB) division of DEC, which had previously designed and built the VAX 8800 series.
The target market for the MSB group was medium-sized businesses: bigger than a mom-and-pop shop with only a few employees, but not a giant multinational company. Able to afford maybe hundreds of thousands for a computer, but not millions. This was a huge, growing market, and a host of minicomputer companies were playing in it, like Data General, Apollo, and Prime Computer. But the big concern on everyone’s minds in the various hardware groups around DEC was almost always “Big Blue” — IBM.
I don’t remember having a lot of discussions about our competition, but there was a de-facto attitude among the engineers I hung out with anyway that the other minicomputer companies, such as Data General and Apollo, were just copycat wannabes — never able to match Digitals accomplishments, and not really a threat. IBM though held a different place in the threat hierarchy for us. I would say they commanded a lot more respect, being the number one computer company, and historically a major force in the development of computer technology.
That didn’t mean we thought they were better than us though. IBM’s image at DEC was one of a stuffy, slow, old-school company who’s time had already come and gone. (Ironically, that was very close to being true for DEC at this point, and in fact IBM would outlive us.) IBM was business-focused, and in a seemingly over-formal, boring way. Their engineers all wore ties, we were told (not entirely true, even in 1986), where at DEC, you would be made fun of if you showed up to work first day wearing a tie, even if you were a manager.
And so I worked in the company thinking IBM was both technically and culturally the enemy. But within the hardware groups that were building mainframe computers, IBM was also the target to be emulated, and hopefully surpassed. DEC was creating bigger and bigger systems, and the once rack-sized minicomputers of the PDP era had given way to larger, multi-cabinet setups, even within the midrange business unit that in theory was supposed to be making smaller, cheaper computers.
I worked for over a year on Argonaut, designing memory controller chips that handled our double-bit error correction and self test pattern generation, and working on the ram array design. We had completed the chip designs and simulations, and were to the point where we were getting close to building hardware.
It was a crazy, incredible time, and I had the good fortune to work with and learn from some very talented people there, including a few who went on to be chip design superstars. Jim Keller and Pete Bannon to name a couple, but if I am going to name drop, I would also need to say that I was a very junior engineer, on the periphery of things. So I rarely interacted directly with the CPU design team and did not know those guys personally too well.
One day though I was sitting in a lab back in Shrewsbury doing going over our design specs with some techs there, and I got a call from the Littleton office. They said management had called a big meeting, and it was about the project and I needed to get back to the office right away. I got in my car and drove up immediately, but it was a short meeting and by the time I got up there, it was over. The takeaway from it was, Argonaut had been cancelled.
I and safe to say a very large majority of people there were completely blind-sided by this, as cancellation had not been even a fleeting thought for us. We were laser-focused on completing our design, and getting it out into the battlefield against IBM. But technology problems had crept in, and the management and some of the senior technical staff there had been quietly trying to resolve a major obstacle that had cropped up with the I/O structure of the gate array chips we were using.
For the technical, this was an issue of simultaneous switching, causing signal integrity problems. It was a big issue, the kind that would normally lead to a major project delay, and possibly some people being fired for making poor technical decisions (but honestly, at DEC we never fired anyone, we were really bad at it)
It could have been fixed, eventually though. Cancellation should have been off the table for a project like this, since building a next-generation business mainframe was at the time, a mission-critical thing for us.
The reason that cancellation of a year-or-more-long effort involving hundreds of engineers was feasible in the minds of DEC’s upper management was this: they had another competing product under development in the company to go to.
Fifteen miles to the south, in Marlborough, Massachusetts, sat a hulking fortress of a building that Digital called “MRO1”. MRO1 was a three-story concrete and glass structure, originally built in 1969 by RCA for use as factory space. It was later purchased by DEC for manufacturing use as well, but eventually converted to office space that was home to a major division of the company - High Performance Systems (HPS).
The sprawling factory floor that had years previously built DECSystem-20s was now a sea of cubicles for the HPS engineering staff, the sheer scale of which was beyond anything I had ever seen, or have seen since. The Argonaut engineers from the much-smaller MSB building in Littleton would fit in one corner of this giant office complex.
This was Digital’s “big computer” division, the group that was tasked with going after IBMs core business with very large, and very expensive mainframe computer designs. And if the MSB division in Littleton had IBM-envy, it was nothing compared with what was going on over in Marlborough. HPS was very much trying to be IBM, so much so that there had been a mini cultural shift from the DEC norm going on there, with managers wearing ties and a general higher level of formality of engineering processes.
I found this out because I ended up working there, next.
After the cancellation of Argonaut, there was a week-long, drunken party at the house of one of the engineers, “Nancy”, who had a place with a big back yard and an inground pool. The day they announced the news we were all in shock, and Nancy came to the rescue, inviting everyone over for an impromptu party — which was a badly needed distraction for all involved. I don’t remember much of it, I think Jim Keller threw me into the pool at one point, (which may have counted as the most significant interaction we ever had.)
It was supposed to be a one day party, but as everyone soon found out, there was nothing to do at work. People would show up in the morning, and the management would give out stuff to read and research, like some things about MIPS architecture that we were looking into. But then everyone would just kind of leave, and we’d end up at Nancy’s house with suntan lotion, coolers and barbeque supplies, probably much to her dismay after two or three days.
The party eventually ended, but things went on over the summer in the same way: you would show up to work, see if anything was going on, management would say “stay tuned”, and you would go home. It amazed me even then that a company would allow this to happen. There were no layoffs, no reassignments to different groups, they just let everything float for pretty much the summer. It must have cost a fortune, and I highly doubt anything would go down like this ever again in today’s tech business world.
My situation that summer was different though, owing to the fact that I was actually not part of the MSB division, but part of Electronic Storage Development. I was a centralized resource, who could be deployed on other projects easily. My managers were not about to let me have the summer off, since HPS was in the middle of their own giant mainframe project, code-named “Aquarius”. I was reassigned to this project, which was also in the late stages of development.
It was not an easy transition for me. Although organizationally, I worked for an outside group, I had been embedded in the MSB design team for over a year and had become quite loyal to our effort, and felt like part of the tribe. HPS to us was not a group we thought about much, but probably when we did, we lumped them into the “IBM” category of being a bloated, overly-formal, stuffy division.
There was not much if any communication or technical exchange between the HPS and MSB design teams, even though we were working on very similar products. So I didn’t know what, if any, transferrable knowledge I gained from working on Argonaut could be applied to working on Aquarius.
Once I got to Marlborough, I found that the HPS engineers were much more prone to looking over their shoulders at what might be going on over at MSB, and did in fact consider the ever-larger machines coming out of that division to be a competing threat to their work. They were all quite happy Argonaut was cancelled, and the product management there expanded the Aquarius product line to include a lower-end variant, called “Aridus” to fill in the gap left by the cancellation of our project. The whole Aquarius program had been envisioned as an “IBM Killer”, but its first (and spoiler, only) victim would be a product within the same company.
And so it was very hard not to feel out of place, working in HPS — almost like I was an exchange student, or perhaps, captured prisoner. I remember listening to HPS people disparage the product group in MSB, and then ask me if the people there did the same kind of thing when talking about HPS. I was the rare guy who had been over enemy lines, and they would often ask about what things were like in MSB, as if we were talking about Russia or something. I kind of shrugged, because honestly we did not spend a lot of time in MSB thinking about them.
Someone should have though, obviously. Today I find it ridiculous that a major corporation would allow huge groups to silo to this degree, to fail to communicate, cooperate, and share, and to allow them to build overlapping and competing products. To allow this kind of animosity to build between employees of the same company. We’re talking about thousands of engineers involved here, between the two divisions. Seems like something that could have only happened in the crazy 1980’s, but I am betting if enough people read this far, we will see comments to the contrary.
This siloed competition phenomenon in fact happens a lot, and I have seen lesser versions of this play out at other, smaller companies I’ve worked for. Sometimes it is just a matter of two groups colliding with each other and trying to do the same thing, or even two individuals. The common goal of beating the external competition gives way to an internal fight for control of something, fueled by lack of communication and sometimes, lack of trust.
Maybe its too simplistic a take, but for me it all of it owes to a failure of management, somewhere in the end. Easy to say I know, and not always easily or effectively addressed, but in essence avoidable. DEC’s version of this failure was just perhaps a bigger example of competitive siloing that is allowed to manifest at many companies, very often to negative effect.
The Aquarius project for its part turned out to be a billion dollar disaster, and that doesn’t even count however much money was spent on our cancelled Argonaut project. Released in 1989 as the VAX 9000, only a few dozen were sold, despite the billion dollar development cost.
Much of this failure owes to technical missteps, to be clear. Aquarius was so-named because it was a water cooled system, a necessity due to the enormous power demands of the bipolar transistor, emitter-coupled logic (ECL) technology used in the system. This technology was in favor at the time due to its speed, but required exotic cooling and interconnect technology that required an army of mechanical and thermal engineers to design.
It all would soon be eclipsed by superior CMOS chip technology, which offered good performance with much less power required. This in turn allowed chips to be scaled up in size, reducing chip counts and thus interconnect delays between chips, boosting the overall performance of the entire system.
And to be fair - our cancelled Argonaut project also utilized ECL chip technology, so would have had similar drawbacks. In the end, the winner (within DEC anyway) in this competitive CPU design contest was neither the Littleton MSB group or the Marlborough HPS group, but what was coming out of our chip foundry in Hudson, Massachusetts. Fast VLSI CMOS processors, starting with the microVAX chip sets but eventually leading to the high-performance Alpha chip, would soon dominate the company, eliminating CPU design groups in both MSB and HPS.
Maybe if we had moved these mainframe projects to CMOS earlier, there would have been a way for both these projects to succeed in the marketplace, who knows. And the larger strategic issue of DEC being slow to follow the “smaller, cheaper, better” route that the industry pivoted to once CMOS technology matured is also a huge factor in our companies decline — and beyond the scope of this story.
It could be that failure was just in the cards for us. When I think back on this time though, I keep returning to the idea that we would have been in a much better position somehow if these company divisions had worked together, rather than competed as rivals. (In writing this, the word “divisions” in the first place seems very tell-tale.)
Certainly the R&D costs would have been lower, and maybe a cooperating set of R&D groups could have come up with a better technical solution. Or if MSB had targeted a lower-end market segment with less overlap, perhaps their project would not have been cancelled, and at least on of the two projects would have succeeded in making a profit. Whatever the case, the whole combined thing had to have gone down as one of the most expensive blunders in computer development history.
I remember in 1993, after it had become clear that Aquarius was a flop, attending an R&D staff meeting in an auditorium in Digital’s famous “Mill” headquarters in Maynard, Massachusetts, where I was then working in the Alpha Workstations group. DEC’s stock price had dropped, we were in the midst of a series of layoffs, and a discontented crowd had gathered to hear Jack Smith, Chief Operating Officer of the company, speak.
Jack was not well liked by a lot of the engineering staff, and had in the past done things such as send out a memo complaining about how the parking lot at the Mill was only half full at 9AM, where in the old days it was always full. When he was not accusing people of shorting their work hours, he was green-lighting bad product decisions, like funding two largely overlapping projects that used soon-to-be-obsolete technology. I don’t remember the content of the meeting, probably a pep talk of some sort, but I do remember the Q&A that followed.
A bold engineer stood up and said bluntly that the company had spent one billion dollars developing a computer that no one wanted, and that if we had instead invested in literally anything else, such as for instance McDonald’s franchises, we would have made more money. Jack (who was obviously bad at reading a room, or otherwise he would not have scheduled a Q&A in the first place) stood there numbly and only said:
“Digital is not in the business of selling hamburgers.”
And then left the stage.
Many other armchair analysists and MBA students have debated the causes of Digital’s downfall, and there is probably no new ground left to cover here. And I don’t know to what degree the unchecked redundancy and inter-division competition played in the failures of these mainframe projects, or of Digital in general. But do I know it certainly didn’t help. For me, the memories of these projects remain as an cautionary reminder of what can happen when people in an organization fail to cooperate with each other.
Shortly after the events of this story, Digital would close its iconic Maynard headquarters, along with many other offices in order to control costs. And in 1998, DEC would be acquired by Compaq Computer, and cease to exist as an independent company.
Postscript: If you read this story, and also my previous post about my time in DEC’s workstation division, you might come away with the impression that Digital was a bad company to work for, overall. Nothing could be further from the truth. In spite of its many failings and the fact it ultimately did not survive as a company, Digital was a great place for engineers, and I have on the whole very fond memories of the experiences I had there.
I received my first patent, for a memory self-test mechanism I designed while working on Argonaut. I made lifelong friends with people on both projects, some of which I have the good fortune to still be working with. I started both my hardware design career, and later my software design career, while at DEC. And, perhaps the best project outcome of all: I met my future wife at Digital, who was an engineer working on the Aquarius project at the time. We were married in 1991, and we’ve outlasted the company that brought us together by over 25 years.
Explore Further
Next Week: Let’s talk demo horror stories and close shaves. I’ve got a tale of misbehaving trade show hardware, straight from Vegas! And I want to hear yours, so lets all put our chips in for: Demo Roulette!
Enjoyed this post? Why not subscribe? Get strange and nerdy tales of computer technology, past present and future - delivered to your inbox regularly. It’s cost-free and ad-free, and you can unsubscribe any time.
Ready for more?
[ comments ]