This post is part of my sporadic Notes to a Young Software Engineer series.
Early in your career, a surprising number of poorly engineered software systems are due to mistakes with engineering media.
In college and at bootcamps, your primary exposure to engineering is through engineering media like Hacker News, meetups, conferences, Free Code Camp, and Hacker Noon. Technology that is widely discussed there — say micro services or a frontend framework or the blockchain — then unnecessarily shows up in your technology stack.
The most common mistake you’ll make is to see these sources as a mirror of what’s going on in our industry. Instead, it’s more akin to a bazaar.
Engineering Media as a Bazaar
Picture a gargantuan bazaar in ancient times. Thousands of merchants fill the stalls, hawking goods from around the world. The bazaar owner finds the most unique merchants, and gives them the most desirable stalls.
This analogy for engineering media works on a few levels. First, the merchants in the bazaar are there to convince you of one thing: to buy their product. In engineering media, the merchants are dev tool companies, training programs, open source projects, and companies trying to hire developers. Given how dependent they are on an engineer’s choices, these are the organizations most motivated to create content and figure out how to get it distributed.
Much like might happen in ancient times, merchants try to become your friends. In engineering, this has similarities to content marketing, where helpful content is used to disguise the advertising. Moreover, many great engineers spend time building systems not writing, limiting their input into important debates.1
In the bazaar, the merchants set the tone. In engineering media, the vocal — those with a particular personal agenda and a less nuanced viewpoint — dominate over thoughtful engineers. Tried and true technologies often have few advocates or rabid early adopters, while the newest have a motivated corp of supporters.
Second, the bazaar attracts buyers of all stripes. In engineering media, Hacker News might attract a startup developer, system administrator, database administrator, data scientist, cryptography engineer, frontend engineer. By determining your technology choices based on popular posts, you can mistake the needs of one community for your own different needs. Even within one community — say web development — the needs of a startup might be very different than the needs of a consulting shop building many websites. Instead, the message shared is that a new technology is the solution to every problem.
Third, the owner runs the bazaar with an eye to maximizing sales, mercilessly favoring the merchants that are successful. In social media and tech blogs, this means increasing engagement and upvotes and page views. In conferences and meetups, this means filling chairs by attracting the right speakers and featuring the content most demanded by the target audience and sponsors. This sets the incentive for every merchant who wants to survive.
Beyond misinforming us, this drive for engagement manipulates our desire to not get left behind. To stay relevant, it seems that we have to use the technologies most hotly discussed. In traditional media, death coverage is commonly used to maximize engagement and get clicks; in software engineering, it is covering new technology.
Free Code Camp Blog
Source: Free Code Camp Blog post, 2014
Compare how your favorite engineering media source might cover topics versus a computer science professor or a fair minded engineer.
Hacker News
Today, Hacker News is one of the more popular watering holes for software engineers.
In Y Combinator startups, like mine, it was an especially valuable tool exposing us to the smartest minds. When used correctly, it let us see the multitude of opinions from the community. It played an especially powerful role for junior engineers and small teams, where a mature community is often missing. But it has a few critical flaws.
As a bazaar, each piece of content depends on how much engagement it gives. If we compare an ancient — but important — build system like Make to Webpack, there will — on average — be much more content and excitement around the newest technology. This can give you a false view of what people actually use in day to day practice. As Paul Graham notes, “The number of … stories about a problem is not a sign of how serious it is, but of how much demand there is for stories claiming so.”
Reddit-type communities like Hacker News often give very little power to experts. Just like a democracy, it gives power to motivated voters, which doesn’t always make sense on highly technical topics. In this environment, the world’s most thoughtful database expert has just a single vote.
Motivated vendors are strategic about sharing their message on these sites through what seem like dispassionate proxies. For example, in MongoDB’s case, it used its fellow VC portfolio companies who publicly proclaimed how much they valued MongoDB. Vendors are also smart about astroturfing. In Y Combinator, we would explicitly create upvoting rings that evaded countermeasures and ask friends to write the first comments to subtly shape discourse.
Conferences and Meetups
Conferences and meetups are the other key way software engineers stay up to date.
Conferences are highly lucrative, but to make them successful, you must focus them on hyped trends for which there is latent interest. This means there might be hundreds of cryptocurrency or React conferences, but few on important tools like C.
Just like blog post authors, many of conference speakers have their own agenda. As MongoDB noted, when crafting their marketing strategy, speakers at their conferences were attracted by:
- increasing the speaker’s network
- raising the speaker’s profile
- recruiting for the speaker’s company
- marketing for the speaker’s company
Note how none of these reasons have anything to do with helping a speaker’s audience make the best decision.
And when a media organization is tied to a regular conference, it becomes much harder to be critical of potential speakers you need to recruit to make the conference successful.
Tips
Perhaps someday we can build engineering media that has the right incentives.
It might encourage thoughtful content and incentivize the community to critique any conflicts. In the case of Hacker Noon or Free Code Camp, it might mean having a board of sober-minded engineers to assess content.
In the meantime, engineering media should be seen as just one data point that should be supplemented by interactions with thoughtfully minded engineers.2 It should not be the primary lens through which you understand what’s going on.
It’s also valuable to critically assess any content put in front of you, by asking a few questions:
- Motive: What is the motive of the writer writing the piece? How does influence its message?
- Background: What is the technical background of the writer? What part of the stack are they effective in? What projects have they worked on?
- Relevance: How relevant are the problems the technology solves to the problems you face? What are the downsides of it?
Relevant Links
This post is part of my sporadic Notes to a Young Software Engineer series.