Monday, October 10, 2011

The Severe Shortage of Skilled and Experienced Engineers

A reader pointed me to this article at October 3, 2011 WBUR:

But according to a new report by the financial services firm, Jones Lang LaSalle, high-tech jobs are growing nearly four times faster than the national average. The report also shows that venture capital is driving the job boom, with high-tech accounting for 50 percent of total venture capital funding over the past year.

Companies say it’s difficult to find and recruit talent, because there is a dearth of qualified engineers.
One comment trying to argue that there is indeed a shortage of skilled engineers says:
We're not looking for specific skills, just someone with current skills and the willingness to learn what we do. (Hey, you do PHP, Python, Perl, Java, that's great...if you want to learn Ruby on Rails too, then we've got a job available now).
 And yet I was just turned down for a job because they wanted someone with PHP experience.  I've written server-side CGI scripts in Perl, and ksh, and I've been writing server-side Java for a couple of years now...but there isn't time for me to learn one more language.  Do you have any idea how many programming languages I have learned since 1971?  So many that I can't even give you a rough guess, but taking off my shoes doesn't help on the count.

Businesses whining about a shortage of skilled engineers are making excuses, hoping that the government will again let them bring in H-1B visa holders to work for $2000 a month.


  1. By the way, Google has announced a new language for the web.

  2. Businesses that can are either outsourcing offshore, if they're big enough, or trying for H1-Bs. There's no shortage of engineers, but there's a shortage of engineers willing to work for what tech companies are willing to pay. And, frankly, age-ism is rampant in this business.

    I'm not sure I'd recommend engineering as a career path until the US gets its industrial and trade policy in order due to all the outsourcing. Our policies are so screwed up that we're shipping out Ph.D. engineers while keeping illiterate migrant workers. And we've been 30 years without a trade surplus, so we're bleeding out our capital.

  3. But PHP is not that difficult. If you know how to program you can do anything in short time

  4. Absolutely right, Clayton. I lost my job 4 years ago to a middle eastern man half my age because of the damned H-1-B visas. I trained him and then they told me I was too old for their "youth oriented corporate culture", despite the fact I had excellent reviews for years and clients loved me because I understand how to work with people in the US.

    He makes less than half what I did.

    I have completely dropped out of the workforce - and no, I never asked for or collected unemployment.

    Now, I raise Angora Goats, Angora Rabbits and hand spin art yarns - I tell people I went from high tech to low tech, and though I miss that money, I'm happy to get the hell out of that corporate culture where employees (though they are called "associates") are considered a resource, just like software or hardware and are expendable any time they can get someone cheaper.

  5. Sounds like you're talking about programming jobs, not engineering.

  6. Engineer doesn't always mean software engineer. We are short competent automation engineers, mechanical, and electrical engineers. No we aren't necessarily hiring H1B folks, but we do hire overseas on our projects because Obambi has caused oil and gas companies with big projects to proceed only with overseas projects. So no domestic projects are pursued until the business environment in the US gets better..... Hence no US hiring. Sorry kiddo.

  7. There is a plethora of great engineering talent in the United States. To large companies, it is only about the bottom line and hiring cheap. Cheap labor usually means foreign labor, either through off-shoring the work or by hiring H-1B visa holders (to include "work study" foreign student visa holders).

    The programming language/skill set argument is a smokescreen. The labor costs associated with the software development environment are the largest single cost in software development. A company can save money and develop code faster using mature programming tools.

  8. I gather there's one more factor in the "hit the ground running" problem you've identified. In general the management of software development nowadays is rather poor and too many managers are uncertain at best about their ability to mentor or even monitor someone who's learning a new language or ecosystem.

  9. I recently read an article in Business Week titled "Software gap - a growing crisis for computers". A few quotes:

    "The overriding issue is people - specifically, skilled computer personnel... Already, the supply is far short of the demand, and the gap is widening inexorably. For the foreseable future, there is literally no possibility that we shall have enough trained people to go around."

    The implication of this gap for business and science ... is that "use of computer systems five years hence will be seriously hobbled". There are only about 120,000 programmers in the U.S. - and right now there's a shortage of 55,000 or more of these new professionals, according to some estimates.


    John A. Devries, chairman of Computer Applications Inc., emphasizes, "Unquestionably there is a software problem that will extend for two years or more."


    Want ads in any newspaper show brisk demand for programmers. Recruiters are active and job-hopping is common; one company is offering a $100 reward for leads to programmers. Salaries for college-graduate beginners start at $6,000; a couple of years experience brings $10,000 to $12,000; special knowledge in key areas such as time-sharing or systems programming brings 15,000 or more.


    "The market for EDP personnel is tighter than hell." According to Dick H. Brandon, head of Brandon Applied Systems, there's a need for 175,000 now - and by 1970 this need will swell to 220,000, plus 275,000 related jobs, in the U.S. alone. And rapid growth in Western Europe makes it an international shortage, as well.

    from "Software gap - a growing crisis for computers", BUSINESS WEEK,
    November 5, 1966, pp. 126-133.

  10. Steve: Never fear, soon after its release there will be a plethora of job listings requiring 2-5 years experience in programming with it....

    I.e. clueless HR departments and criminally stupid management that surrenders critical functions like computer systems and programming recruitment to it are also a significant part of this problem.

  11. As a 50-something retired Intel SW engineer I sympathize with "oh no, not another language/API" feeling.

    However PHP can be considered a simplified, syntactically sweeter variation on Perl and Java, especially when doing database, XML, and web chores. Of course, a PHP job probably also requires familiarity with at least one of the major content management systems, like Wordpress, Joomla, or Drupal.

  12. My experience is that I have been getting calls for software positions (usually in/around Metro Detroit, so mostly in-Auto-Industry) on a regular basis.

    I just changed jobs, and am settling in (with a 15% salary increase). So I'm ignoring them.

    But there's a demand, and it's apparently the only kind of work for which demand for workers exceeds supply.

  13. Come visit Houston's Aerospace Transition Center. About once a week is an engineering hiring event in which several hundred engineers with background in material science, electrical, mechanical, software, and operational research are looking for a job. Experience ranges from a few years to several decades.

    True, many of them don't prefer to move. However, some are talking about jobs in foreign countries.

    Bottom line, I don't buy the notion of a shortage of engineers. While shortsided, too few companies/industries can afford to redesign or develop new products to market in this economy.

  14. The lies about H-1B recipients undercutting local workers are just lies. I hereby challenge previous commenters SWWBO and Gaston to hame names, markets, and salary amounts.

  15. 1. Businesses whining about a shortage of skilled engineers are making excuses, hoping that the government will again let them bring in H-1B visa holders to work for $2000 a month.

    Clayton, you're being far too mild. My euphemisms of choice are terms like "lying vermin". (Iirc William F. Buckley remarked that the trouble with socialism is socialism but the trouble with capitalism is capitalists.)

    2. Columbia mathematician Peter Woit blogged skeptically about the "shortage" back in 2010 and 2004. In the 2010 post I commented that the National Science Foundation, headed by an IBMer, was pushing this BS back in the 1980s.

    3. Most of the links in gadfly entrepreneur Philip Greenspun's site are dated, but their content remains relevant. Sample quote: I am fascinated by the 30-year decline in the relative salaries and prestige of engineers and scientists that has been accompanied by 30 years of statements by politicians and university administrators that there is a shortage of engineers and scientists. The day may come when these politicians and "educators" (and businesses) bring the fate of the boy who cried wolf on themselves--and, unfortunately, on the whole country.

    4. Science and engineering are intrinsically satisfying and I do not discourage people from entering these fields--iff they clearly understand that idealism or craftsmanship attract attempts at exploitation by amoral manipulators.

  16. Last month, there were over 1,700 ads for Java programmers in the DC area. 300 in Atlanta, ~300 in Raleigh, and similar numbers in other East Coast cities. I know this because I have been trying to fill positions since January. After nine months, we finally hired our first ever H1B candidate to fill one of the positions. I still have another one to go, with a possible third position opening up.

    Clayton, there are definitely jobs demanding your skill set, but they may not be in Idaho. Are you willing to move to get them? Because the immigrants are?

  17. As to the cost of H-1B hire. Don't assume that they are working for $2000/mo. The law the governs the H-1B process only specifies direct payment.

    So the scheme is the Mega Corp goes into contract with Inda Outsourcer. The deal that goes down is that for each placement Corp pays the agreed rate + a `management fee`. Outsourcer skims some off the top then the balance of the fee is paid in local currency of the person placed.

    Mega only has to report FICA on the $24k/annum that is paid in the US. The balance since it is treated as G&A expense goes against the bottom line, no FICA collected. Another words if the total compensation for person X is $40k, Mega only pays FICA against $24k of it. Reducing Mega's total labor cost yet again.

  18. A contrary view:

    The market for temporary (6-24 month contract) IT work has definitely improved in my region (the midwset). I'm seeing hourly bill rates climb very nicely indeed, and have already exceeded my pre-crash hourly bill rate.

    I only worked 4 months in 2009, but then completed a 12 month contract in 2010, and went immediately into a new contract in 2011. I do have 25 years of software development experience though, so your results may vary.

  19. Been an engineer for 30 years. I've been listening to the companies and pols whining about "engineer shortages" even more years than that - all in service of bringing in more low paid imports.

  20. I am not afraid of PHP--it seemed like yet another scripting language to me, not much different from many others that I have written.

    If I were unemployed, relocating would make sense, but I have a government job at the moment. The pay is awful, but by the time I relocated, the net result would probably be about what I make now!

    What upsets me is the claim of this severe shortage which does not seem to fit with the experiences of software engineers that I know.

    Regarding the comment above by someone whose name showed up as Kanji that H-1B visas are not undercutting local workers: by law, employers are required to pay comparable wages. What I have seen of how companies like HP work is that they lay off experienced workers while simultaneously seeking to hire very generic software engineers with five years of experience--replacing $100,000 a year engineers with $50,000 a year engineers. See I can't really blame HP and other employers for doing that, because of competitive pressures, but there's no reason for our government to approve H-1B visas for such a purpose.

  21. Here's a link to a previous example of HP trying to hire H-1B visa engineers while engaging in a massive layoff:

  22. There are good programming jobs out there, but not a lot of good programming jobs out there for folks who live in Idaho and who aren't willing to relocate. As an employer I either want to build an intact team at my location, or I'm willing to save money by outsourcing/offshoring and am looking for the low dollar employee. You don't want to compete with the guy in Bangalore, so you'd better have a rare, specific technical skill that is in high demand. Otherwise you're going to have to go to where the jobs are. Probably not what you want to hear... but it's the truth.

  23. I hereby challenge previous commenters SWWBO and Gaston to hame names, markets, and salary amounts.

    Well, I'm not one of them but I most certainly can:

    Name: Lucent
    Date: 2001
    Market: Metro D.C. (NoVA)

    Salaries: I was earning $80K while a very smart and in many ways more qualified Jamaican was earning $48K for the same job (the law apparently requires posting the H-1B's salary at the workplace).

    I don't think it was a coincidence that as Lucent downsized that year I and another American citizen (with a family and plenty of experience so I presume he was earning quite a bit) were purged in favor of the H-1B. The latter certainly felt he was being exploited....

  24. Last month, there were over 1,700 ads for Java programmers in the DC area.

    If you're actually an employer in the IT game, then I'd expect you to know that the great majority of those "jobs" on dice do not actually exist. They're resume bait for consulting companies.

  25. Mr. Clifford:

    I am well aware that being in Idaho is a real disadvantage, and from that standpoint, if an employer has to pick someone cheap in Bangalore or someone less cheap in Idaho, I can see why he would choose someone in Bangalore. After all, his communication skills will be every bit as good as mine, and there are no time zone difficulties for an American company to talk to Bangalore. :-)

    What annoys me is when companies bring H-1B visa works here on the pretense that they can't find anyone to do the job here. This is not a true statement. I am even more upset by the fact that many U.S. employers increasingly would prefer to hire someone with two years of experience for the same salary as someone with thirty years of experience. I can believe that beyond a certain point, more experience does not provide any significant economic benefit. But does more experience really make us worth less, or worse, worthless?

  26. Mr. Clifford:

    Let me throw another possibility at you. If an employer is prepared to offshore a job to Bangalore, why would they refuse to onshore a job to Idaho? When I worked for HP, I was well aware that the days of $1200 a month Indian software engineers were over. HP started to move its offshore development work from Bangalore to Shanghai because of costs--and by some projections, Shanghai development costs were going to be reach Idaho development costs around 2014.

    What are typical offshore development costs now, keeping in mind that there is more than just the salaries:

    1. Cost of shipping development hardware offshore. (There was a startling amount of money spent for HP to ship early pre-production laser printers and testbeds.)

    2. Cost of phone calls and airfare for the inevitable face-to-face meetings and time spent training the offshore developers.

    3. The time and opportunities lost because of time zone turnaround. Monday, someone in North America sends an email requesting information; Tuesday, someone in Bangalore responds to the email. Wednesday, someone in North America realizes that the question asked was not the question answered, and sends a clarifying email. Thursday, the person in Bangalore responds with the information that was actually needed.

  27. If you're actually an employer in the IT game, then I'd expect you to know that the great majority of those "jobs" on dice do not actually exist.

    I'm familiar with the resume piles needed for winning government contracts. That's why I included the other cities and the other information. You seem to have conveniently glossed over that.

  28. With regards to hiring a remote worker in India/China/Russia or Idaho, I'd prefer to go with Idaho. It's easier to get a domestic remote worker into the office for face to face meetings and training, and even scheduling conference calls is easier when the timezones are so skewed.

    I've had two bad experiences with off-shoring (neither were my idea). But I will note that the second one I ended up handing to an American contractor who then failed to finish the job and left us high-and-dry for a better gig. I also had another local American contractor do the same thing on a different project.

  29. Clayton's longer comment touches upon two important points.

    The lesser one is, how much change did the hiring see and how much fraud does H-1B see these days. Back when I received my H-1B, my salary was $75k/y with benefits, which ok for Silicon Valley, depending on the responsibilities. I hardly spoke any English, too. Of course if they hire people for $48k/y today, when dollar is perhaps 50% weaker, that may be concerning.

    The second part is, programming is even more stratified today than it was back in Brooks' days. If you have to hack together PHP scripts, you are expendable, and Clayton really should consider himself lucky not to get into that sweatshop. Those $48k are actually quite a lot for people doing that, and $80k would be absurd! On the other hand, my company is suffering while trying to find qualified people at the top. Many candidates, few qualify. We continue to do H-1Bs for those who still consider it a perk (e.g. Matthew "mjg69" Garrett), we sponsor Green Card for those who need it (I think another kernel guy, Prarit Burhava, was on it), but most of the time we hire internationally these days.

    This north-south division may help to explain how companies lay off hordes of low-qualified programmers, simultaneously with searching for good ones. Unfortunately, I cannot tell if HP is anything like that: the company was quite mismanaged even before Carly, and Mark did little to right it. They may simply be in the throes of collapse now.

    The H-1B was supposed to be the vehicle to get the top of the parfait, but the practice steadily expanded to the bottom, as the quality of the candidates fell off. I do not see an easy fix, as most of the commentary seems to be of the griping variety. I am not even conviced that we do have a problem. And if we do, I do not think that merely bumping admission requirements is it, primarily because credentials are often unrelated to the capability of individuals in our field. Perhaps jacking up fees for H-1B applications would help.

    -- Pete Zaitcev

  30. There are times when H-1B may be perfectly legitimate. Some years back, I was trying to hire software engineers in Mountain View--and in 1985, this was really, really hard. We ended up hiring a South African on an H-1B visa (who of course became a multimillionaire at a startup that I did not go to, to my regret) because we really could not find anyone willing to move to Silicon Valley, and there was no one there. We were paying well, and our requirements were very minimal.

    That would not be the situation today.

  31. The problem is primarily, but not exclusively, a computer engineering problem.

    When I worked for Dames & Moore (a large geotechnical engineering consulting firm, since bought by URS), there was a relatively fresh staff engineer working there who was a foreign student who had very recently graduated. (I was finishing my degree while working there). One day, an ad appeared in the paper, looking for a staff engineer. The ad was *obviously* tailored to the foreign student, as it required experience with in-house software and equipment that nobody else would qualify for.

  32. There is a new update to the original story at

  33. Then there is the problem of not being able to move because of an underwater mortgage (my problem)...if only I had stopped making payments instead of paying over $100K to principal and now I would see zero of that back were I too sell my house...I have had to forgo job interviews because of that. One of them was with Apple last year...

  34. W:

    This is the problem with a system that is increasingly set up to penalize those who do the right thing.

    The only good thing about my situation is that the house payments are not horrendous on my house. If I had to move somewhere else to work, it would be a nuisance to leave the house vacant, but it would not be a disaster. Many other engineers do not have that choice.

    Cleaning up the housing disaster will probably do much to improve mobility for high-skill and high-paying workers.

  35. I am not a fan of offshoring; companies usually offshore to save money and that almost never works. However, if you wait for US tech companies to get rational, you may be waiting for a while. The accountants are still in charge when it comes to hiring. Re H1Bs, too many tech companies still believe that engineers are fungible... why pay more for the same capabilities? (I don't believe this.) Others take advantage of the fact that what is exorbitant pay for some foreign workers is very low pay here in the US. For others, the overall cost and risks of domestic employees are higher than offshoring, and again they're focused on the bottom line. And finally, today's job market is an employer's market... and that sucks for most of those looking for a job. I left a state where the quality of life was great and the cost of living was low to move out to the West Coast in order to get a job... I'd like to move back but jobs are here, not there.

  36. I understand that it is not a factor "to hit the ground running" problem you have identified. In general, the management of software development today is pretty bad, and too many managers are uncertain at best their ability to mentor, or even to follow someone who is learning a new language, or the ecosystem.

    hp salaries