Developers are faced with a mountain of complexity, and it's only getting worse.
When we decided to build Flightdeck, our internal developer platform based on Backstage, we did so knowing that developers today face a mountain of complexity, even for the most simplistic of applications. Between an ever-growing list of dependencies, deployment environments, and even third-party services, it is increasingly difficult for individual developers maintain a complete view of their software estate.
And, this problem is further compounded once team dynamics are introduced. Questions like "how is knowledge transferred?", "what happens when a maintainer leaves the organization?", and "how will be bring new developers up to speed quickly?" are inevitable.
While this phenomenon was something we were already keenly aware of, it has been an acute concern of mine in particular, and especially recently. Even as the creator and lead developer of our platform, there were many times where I had forgotten what I had done, or even how things worked as complexity increased. Keeping track of repositories, systems, and tools was a constant struggle.
How complex was it though? I decided to take a look. I knew that our platform had a handful of frameworks, dozens of managed services, thousands of dependencies, and hundreds of thousands of lines of code. (This doesn't even include the massive Backstage code base!)
craig@bol:~/src/github.com/arctir/meta$ ./cloc.sh
1175 text files.
1099 unique files.
180 files ignored.
github.com/AlDanial/cloc v 1.90 T=0.79 s (1314.0 files/s, 270729.7 lines/s)
-------------------------------------------------------------------------------
Language files blank comment code
-------------------------------------------------------------------------------
Go 254 15884 6362 66757
JSON 44 1 0 39322
YAML 220 677 1648 38584
TypeScript 189 1423 2735 11804
JavaScript 96 1157 1368 6850
JSX 103 537 161 6551
HCL 71 805 424 4033
Python 14 248 124 1171
Markdown 13 513 0 1086
SQL 5 15 115 697
HTML 3 463 0 472
make 7 86 35 337
CSS 6 58 26 222
Jinja Template 1 8 0 129
TOML 1 7 1 54
Dockerfile 2 17 13 30
Bourne Shell 3 4 0 21
SVG 1 0 0 10
XML 1 0 0 3
-------------------------------------------------------------------------------
SUM: 1034 21903 13012 178133
-------------------------------------------------------------------------------
What will this look like as we grow our team? In prior roles, some of the biggest roadblocks were directly related to the complexity of the code base, discoverability of the environment, and organizational changes.
The only way that this is remotely manageable is with new, modern, cross-cutting tools. We are dog-fooding Flightdeck because, like you, we want to be more efficient with how we deliver software; making it simple for new developers to come onboard and providing meaningful insights to those who have been around for a while.
It's our head-up display. Everything we need at our fingertips.
Even though we are still small we aren't alone in these problems. Development teams often have huge numbers of repositories, massive numbers of dependencies and while we, as an industry, understandably care about the security implications of these issues, we never fully focus enough on the impact to developer experience that this causes.
We are changing that.