When I graduated, being a software engineer mostly meant writing code. Today a lot of that code gets written by LLMs. Instead of my Senior Engineer Slacking me to look into something, they're typing a prompt into Claude.

Ease of writing code doesn't make it less important; if anything, it makes its safety more important. Autonomous systems handing software the authority to make decisions that a human used to make has great real-world effects. When that software is wrong, the result isn't a 500 error or a misshapen UI element. It's a collision, a fire, or someone losing a hand.

System safety asks how a system can hurt people before it gets the chance to. I want to bring a software engineer's eye to it: someone who knows how code actually fails, not just how the requirements say it should work.

Independently, I've studied MIL-STD-882E and the Joint Software Systems Safety Engineering Handbook (JSSSEH), and worked through three case studies end to end. This post explains why I'm making the move and walks through one of them: a hazard analysis of an extra-large uncrewed undersea vehicle.

My Perspective as a SWE

I've worked as a software engineer for six years since college. Before that, internships had me doing full-stack development, data science and DevOps. Along the way I've done the work that teaches you how software breaks: code reviews, concurrency, and chasing defects that only appear under the right conditions.

The one I still think about was at a logistics company. A client started using a feature nobody had touched in years, and under certain conditions it produced dates that were badly wrong. The cause was old date-handling code that had never been made to work past 1999. It was a Y2K bug that had been sitting in production for years, because the conditions for it to happen were never met until years later.

Finding latent failures before they matter is what hazard analysis is for. A bad date on a shipping report is an annoyance or a missed package, but on an autonomous system it puts dozens of lives and an expensive vehicle at risk. Race conditions, stale caches and a message marked "sent" that never arrived are everyday software problems. In a safety-critical system, they become causal factors in a hazard.

Case study: TIDEWATER, an extra-large UUV

TIDEWATER is a fictional program I used to practice the full MIL-STD-882E process on a realistic, software-heavy autonomous system. Practicing engineers in the field reviewed it for realism. It isn't based on any real program, and every figure is invented.

The system

TIDEWATER is an uncrewed undersea vehicle (UUV) that runs for up to 21 days, surveying the seabed and deploying sensor nodes. A shore Mission Control Center supervises it, but only hears from it at check-ins up to 24 hours apart. For most of a mission, the vehicle is on its own.

The case came with a test history of five incidents. A false battery alarm triggered an emergency surface in a boat channel. In simulation, "keep schedule" outranked "avoid obstacle." A dive plan in feet passed review as meters. An Abort command showed as sent in the shore app for six hours but never arrived. A simulated seabed cable was missing from the exclusion-zone cache. Nobody had analyzed the software hazards.

1. Preliminary Hazard List. Identify hazards across every life-cycle phase: test, load-out, launch, transit, check-in, maintenance, recovery, transport and disposal.
2. Preliminary Hazard Analysis (Task 202). For each hazard, write the source, mechanism and outcome; list causal factors across hardware, software, human, environment and interfaces; assign initial risk using 882E Tables I to III.
3. Software criticality. Give each safety-significant software function a software control category (SCC) and software criticality index (SwCI) from Tables IV and V, which set the level of rigor in Table VI.
4. Design decision assessment. Trace 11 program design decisions (D1 to D11) to the hazards they create or worsen, and recommend accepting, mitigating or rejecting each.
5. Mitigation. Apply the design order of precedence: eliminate, reduce by design, add engineered safety features, add warning devices, then procedures and training. Reassess target risk.
6. Thread and fault tree analysis. Follow the emergency-surface function end to end with a safety-critical function thread analysis (SCFTA), and build a fault tree for the surfacing-collision hazard.

PHA Findings

All 11 hazards started at High risk. Every one had a catastrophic worst case, and probabilities were high because the design relied on software alone, with no independent check.

For the purpose of the case study I singled out three hazards to perform mitigations on.

Three hazards up close

H-01: The propeller turns during maintenance. Maintenance mode was a task that could be sent from any shore console or the pier laptop, and nothing physical at the vehicle stopped the propeller (D8). A sailor with their hands near it was relying entirely on software being right. The propeller can't be eliminated, so mitigation starts with design: make maintenance mode the default state, which can only be inhibited or exited, and cap propeller speed while in it. There were reports of the decision tree erroneously changing the tasked behavior; that shouldn't be a possibility in this situation. A safe maintenance state should be the default. The engineered safety feature is a hardwired interlock with a physical key at the vehicle, independent of software. Lights and a tone mark mode changes, and signage and rehearsed procedures come last. This brought the target risk to 1D: Catastrophic and Remote, Serious, down from the original High risk.

H-02: Feet versus meters. The Mission Planner converted chart depths in the wrong units, and the plan passed both human review and the validation simulator (Incident 3). The planner was classed as ground support (D7), so nobody treated it as safety significant. The vehicle didn't check depth before diving, and its runtime monitor shared a processor with the software it was supposed to watch (D4). The fix makes units a required part of plan data instead of bare numbers. The first fix is a check at the front end of the software. The user shouldn't even be able to task a depth greater than the specified operating depth. Rejection and the inability to surpass the specified depth should be the first line of defense. The vehicle validates every plan at load against a hardcoded 1,000 m limit in the backend as this data gets sent to the UUV, and an envelope monitor enforces the same limit at runtime. There was no stated envelope for stopping, and stopping an underwater vehicle at the max depth is not feasible, so there should be a smoothed stop as it approaches the limit. At tasking, plans past the limit are rejected via front and backend validation; plans close to it get a warning. This brings the target risk to 1E: Catastrophic, Improbable, and Medium risk.

H-04: Surfacing into a ship. The daily ascent cleared the surface using passive acoustics and cached vessel tracks that could be hours old (D3). Emergency ascents skipped the check entirely, and a single false alarm from the vendor battery management system had already caused one in a boat channel (Incident 1, D9). The vehicle's AIS and strobe stayed off until it was fully surfaced. The fix starts with a rule: stale data can stop an ascent, but never permit one. The vehicle ascends to an intermediate depth and listens before surfacing. More than one sensor has to agree before an emergency ascent. Lights and AIS come on during the ascent, trading some covertness for not hitting anyone. Target risk: Catastrophic, Remote, Serious.

Software criticality

In 882E, the software control category describes how much authority software has over a hazard. Combined with severity, it gives a software criticality index, which sets the Level of Rigor tasks to analyze the software for a given software system. Almost every TIDEWATER hazard was SCC 1 (autonomous) with catastrophic severity. That's SwCI 1, the highest level of rigor, with analysis all the way down to the code.

The mitigations for H-01 move the software from SCC 1 (Autonomous) to SCC 3 (Redundant Fault Tolerant) by adding independent ways to detect and intervene: a physical key, an onboard validator, multi-sensor agreement. Those functions drop to SwCI 2. The goal isn't to test less. It's to make sure software is no longer the only thing between a fault and a mishap and increase the influence that software alone has for that hazard.

Following one thread: emergency surface

This conceptual SCFTA traced the emergency surface function from detection to the shore finding out. Every step had a weak point.

Step 4 is one a software engineer spots fast. Two identical computers running the same build protect against hardware faults, not software faults. The fault tree for H-04 showed the same pattern from the other direction: stale track data shows up in both of its main branches, so it's a common cause worth fixing first. Multiple VMCs create physical redundancy, but if the software fault exists on both, the software fault still exists and is not any less likely to happen.

Design decisions

Of the 11 design decisions, I recommended rejecting three: the software-only drop weight (D2), surface clearance from cached tracks (D3), and maintenance mode by remote task (D8). I found these to be unnecessarily high risk given the nature of the system. Seven needed mitigation. I accepted one, the 14-day expiry on cached exclusion zones (D6).

What the case study taught me

Most of TIDEWATER's hazards weren't exotic. Read down the causal factors and the same software problems keep coming back:

• Stale data treated as current. Cached vessel tracks, positions extrapolated across 24 hours, a vehicle silent for 72 hours shown as normal. It's layers of an autonomous system making decisions right now based on data from days ago.
• Redundancy that isn't. Two computers on one build share every bug. Physical redundancy does not mean software redundancy.
• A UI that shows assumptions rather than outcomes. "Sent" isn't "received." Without an acknowledgment from the vehicle, the Abort button was a hope.
• Safety-critical code hiding in support software. The Mission Planner never runs on the vehicle, so nobody treated it as safety significant, yet it could send the vehicle past its maximum depth.

All in all, this was an interesting thought exercise, and reading and learning the literature behind systems safety showed how a native software engineer can intuit fixes for autonomous systems. As software continues to make decisions that influence vehicles and the world around them, I foresee software systems safety, and systems safety as a whole, being a much larger and more impactful niche within engineering.

Interest in Crypto investing is at an all time high. As of this writing the entire cryptocurrency market capitalization is valued at over $1 Trillion USD with thousands making their first investment every day. Cryptocurrencies and blockchain technology have been an area of interest of mine for close to a decade. With all the newcomers to the space and it’s domination in the news cycle the question always gets asked “What cryptocurrency should I invest in?” The short answer for about five years has been more or less “Mostly Bitcoin and Ethereum, but look into [insert new hot project here]” The thesis being Bitcoin and Ethereum are two projects that will stand the test of time but you should expose a percentage of your portfolio to new emerging projects. While it’s clear that this has worked let’s see if the data backs this up.

First and foremost, we must select a collection of cryptocurrencies to test this with. I stuck with coins in the top 50 of market capitalization, have been through at least one previous market cycle (this meant coins quality and highly performing coins like Cardano’s ADA, Chainlink, Binance’s BNB, Polkadot, and others could not be included due to lack of historical data), and I didn’t include “meme coins” like Dogecoin due it’s appreciation being more in part to anomalies than providing meaningful utility. I ended up choosing Bitcoin, Ethereum, Ripple’s XRP, Litecoin, and Stellar Lumens for this test.

To find the “best” portfolio we need to apply a mathematical model to the historical price data. There’s no perfect model but I’ve applied Modern Portfolio Theory to the data set. Historical price data is pulled from the coingecko.com API due to it being free and easy to work with. Modern Portfolio Theory was introduced by Nobel Prize winning economist Harry Markowitz. MPT doesn't look at a given asset's risk-return but rather how that one asset effects the entire portfolio's risk-return. This allows one to construct a portfolio with the biggest return given an accepted level of risk. An additional metric to examine is the Sharpe Ratio. The Sharpe Ratio was made by William Sharpe and assigns a number to a portfolio give it's return subtracted by the risk free rate, divided by the standard deviation of the portfolio's excess return. In short, a high Sharpe Ratio will give a better return than a portfolio with a smaller Sharpe Ratio. Using these two we can see what would be in theory the optimal portfolio.

What did we find? In the portfolio with the high Sharpe Ratio the breakdown is as follows.

The portfolio above is the highest performing data point on the graph above. 68% of the portfolio is allocated to Bitcoin and Ethereum. One reason for this is their strong growth for the past 4 years and perhaps more importantly, their relative lack of volatility. While both BTC and ETH have both experienced 1 year drops of 80%+ that is not as great as the 90%+ drops many altcoins have experienced. When optimizing your portfolio, one lying on the efficient frontier represents the combination offering the best possible expected return for a given risk level, you take into account both returns and the downside risk. One thing I found surprising was the low percentage of XRP. XRP has a massive following with many investors excited about their future and it does reflect XRP’s general price instability.

What does this mean for the new crypto investor? Bitcoin and Ethereum should comprise the majority of one's portfolio. Many coins and tokens can have massive gains and a $5 price per token looks much more enticing than Bitcoin’s current $40,000 price but the data speaks for itself.

For as long as I can remember I’ve been a big fan of basketball. I enjoy playing basketball, watching basketball, and following and debating the NBA. One of the biggest conversations in basketball is comparing different players to each other and who should be awarded certain accolades. This most prestigious of which in the NBA is the Most Valuable Player award. This is awarded to whom the voters deem as, you guessed it, the most valuable player in the league. Growing up when debating one player over another I would always use stats to back up my arguments. So when I decided to take the wealth of historical NBA data and see if I can use a machine learning model to accurately predict which players would win the MVP award.

The first challenge I ran into was determining how to decide if a player wins or not. Looking deeper into the process for determining MVP I saw that 121 selected voters were to assign players votes ranging from first place (most valuable) to fifth place (fifth most valuable). Historically, it comes down to who gets the most first place votes. There hasn’t been an instance of a player winning the award from getting a lot of 2nd place votes. So I decided to predict the amount of first place votes a player would get.

The next challenge I faced was which statistics I would use to predict the number of votes. Using basketball-reference.com to collect the stats, I took every player that received votes for MVP in the past 10 years and recorded their advanced statistics. I chose advanced statistics because they can give a better indication on a player’s performance and impact on the game. In addition to this I didn’t base predictions on the players’ raw stats rather I chose their rank in the league for the year. Reason being, the MVP award is given to the most valuable player for that year. Just using the raw numbers would have the model make predictions based on every players performance in the history of the league rather than against the performances of other players from that season. Doing this I had the necessary to make predictions. After performing a feature selection I decided on using three different statistics: PER (Player Efficiency Rating), VORP (Value Over Replacement Player), and WS (Win Shares). PER was a statistic made by John Holligner which “sums up all a player’s positive accomplishments, subtracts the negative accomplishments, and returns a aper-minute rating of a player’s performance.” As per Basketball-Reference.com, VORP is “an estimate of each player’s overall contribution to the team, measured vs. what a theoretical “replacement player” would provide.” The top 10 seasons all time for VORP is a list populated by Michael Jordan, LeBron James, David Robinson, and Chris Paul, all players who have finished at least top two in MVP voting. Basketball-Reference.com defines Win Shares as “Win Shares is a player statistic which attempts to divvy up credit for team success to the individuals on the team.” With all of these stats available I was ready to begin doing some analysis.

When choosing an algorithm to predict a player’s received votes I chose to a regression algorithm rather than a classification because the number of votes received is a continuous numerical result rather than a discrete one that would be better suited for a classifier. I began with a simple Linear Regression.

A simple Linear Regression posed two problems for this data set. One was that the algorithm only accepted a single input variable where I had many and two it had rather mediocre performance. With a Root Mean Squared Error of 11.87 I knew it was possible to get better performance. In an attempt to better utilize the data I decided to use a multi-dimensional Linear Regression. This algorithm works similar to the previously used one however it uses multiple input variables. Using all three input variables rather than one I got a slightly better Root Mean Squared Error.

The weakness of Linear Regression in this application is the data used isn’t linear as demonstrated by this graph of Votes and PER

Because of this I then decided to go with a good algorithm for non-linear data sets: a random-forest regressor. The way how random-forest works is by creating a number of decision trees and using the mean of the trees to predict the outcome. Random Forest works for this application because it is a non-linear model. When running a random forest I got an error of 3.46 votes which was far better than what I got with a linear regression. To test how it compared to real life MVP voting I had the model predict who won in 2017, a highly contested year where many people were split between James Harden and Russell Westbrook. The model said Westbrook should have received 79 votes, more than the 69 votes he received in real life.

Application of historical data to modern basketball is both interesting and controversial. While in my opinion analytics can be very telling to gauge a player's performance some neglect the "eye test" and intangibles that makes a player great. There are some advanced statistics that translate well between between different eras of the game that quantify player performance and their contributions to a team's success and will stand the test of time as indicators. Ways to improve this could acquiring more data (a decade of MVP voting was used in this test), using different sampling methods, or using different stats such as those quantifying team performance. Revisiting later for other awards and future awards may yield interesting results.