SavvyAI logo

FEVA™ 2.1
Predicting exhaust valve failure with engine data and machine learning

FEVA 2.1, the latest version of FEVA (Failing Exhaust Valve Analytics) introduced in May 2021, provides a 20% improvement in predictive accuracy.

An exhaust valve failure can be an expensive and dangerous event.  Since 2014, the FEVA (Failing Exhaust Valve Analytics) feature of SavvyAnalysis has scanned uploaded data from more than three million GA flights for the telltale signs of a burned exhaust valve whenever engine data is uploaded to the platform.  On more than 60 occasions, FEVA has caught a failure before it happens.

Then, in early 2020 Savvy rolled out FEVA 2.0 to SavvyMx, SavvyQA and SavvyAnalysis Pro clients.  Leveraging data from over 3 million flights, FEVA 2.0 was a significantly improved and more accurate version of FEVA.  It was the first practical application of machine learning technology to GA maintenance.

Now FEVA 2.1, the recent upgrade of FEVA, incorporates more predictive variables from an aircraft’s entire recent flight history, resulting in more accurate, sensitive and consistent predictions.

To learn more about this exciting technology, see the FAQs below.

In the above report, FEVA 2.1 has predicted that cylinder #2 is predicted to have an above-average risk of exhaust valve failure. We would strongly recommend doing a borescope inspection of the #2 cylinder as soon as possible to determine the actual condition of the exhaust valve.

General questions about FEVA

Failing (burned) exhaust valves are “rare events” but they’re the leading cause of cylinder replacement, and exhaust valve failure is a significant cause of piston engine power-loss incidents. Compression leakdown testing has traditionally been used to identify failing exhaust valves, but it is an imperfect test at best, and requires spark plug removal. Visual borescopy is a much more reliable method of evaluating valve health, but it also requires access to the cylinder innards through the spark plug hole. Wouldn’t it be great if there was a non-invasive way to assess valve health? Enter FEVA—Failing Exhaust Valve Analytics.

Digital engine monitors, now ubiquitous among small GA aircraft, provide a window into the combustion process and the health of the cylinder itself. We’ve long observed that failing exhaust valves often cause small, slow, rhythmic EGT oscillations that are fairly easy to spot in a graphical plot of the data. In 2015, we developed FEVA, a heuristic algorithm that scans all uploaded engine monitor data for this signature. We’ve been using this to alert owners to the possibility that they may have a failing exhaust valve and that a borescope inspection of the cylinder is suggested. But to be honest, we were less than thrilled about the accuracy of the FEVA algorithm. That led to the development of FEVA 2.0, introduced in Spring of 2020, and now, FEVA 2.1.

FEVA 2.0 was the improved version of FEVA released in Spring of 2020. FEVA 2.0 used machine learning technology and an expanded data set to predict failed exhaust valves, and presented the results in a new way.

FEVA 2.1 uses the same machine learning technology as FEVA 2.0. But FEVA 2.1 uses a larger set of predictive variables than 2.0, and includes data from an aircraft’s entire recent flight history.

FEVA 2.1 is included in all SavvyMx, SavvyQA, and SavvyAnalysis Pro plans at no additional cost.

Mike Busch has written and lectured extensively on the subject of exhaust valve health. Here are links to his articles, webinars and his book on the topic:

Articles:

Webinars:

Book:

The book Mike Busch on Engines also has an extensive discussion of exhaust valves.

FEVA 2.1 technology

FEVA 2.1 employs modern machine learning technology to predict the probability of valve failure. Machine Learning is an application of artificial intelligence (AI) that provides systems the ability to automatically learn and improve from experience without being explicitly programmed. Thanks to engine monitor data from over 3 million flights in our system, and valve condition data gathered in our maintenance management and consulting services, our software is “learning” what patterns in the engine monitor data predict the risk of valve failure.

The original FEVA made predictions about a cylinder’s health using only the EGT trace from a particular cylinder for a single flight. FEVA 2.0 combined over 30 different measures derived from digital engine monitor data to make its predictions. FEVA 2.1 employs even more predictive measures from a larger set of an aircraft’s recent flights.

Yes it does. The rhythmic EGT oscillation is still one of the predictors in FEVA 2.1.

The FEVA 2.1 model includes over 35 variables, mostly from your engine monitor data, to make its predictions. It looks for hidden patterns in the data — patterns that might not be apparent to the human eye. It’s very difficult to identify which specific variables are causing the model to predict higher failure probabilities for certain cylinders than for others because the model is a very complex function created by an ensemble of decision trees. We use this “random forests” machine learning model because it is quite powerful for the number of training examples we have. The drawback is that it is somewhat hard to tease out the effects of individual variables.

When an exhaust valve failure does occur, Savvy’s clients often ask us “What did I do wrong? How could I have prevented the failure?” The answer is: You probably did nothing wrong, and you probably could not have prevented the failure. We have found that exhaust valve failure is caused primarily by factors outside the control of pilots and owners, such as variations in assembly tolerances and materials of cylinder assemblies. However, if you would like us to review your powerplant management technique using data from a specific flight, simply request analysis of the flight in the normal way and note that you would like us to focus on your operating technique.

When we test the FEVA 2.1 model against a set of test data we find that FEVA 2.1 is able to predict the valve failure risk category with a high statistical significance. Additionally, FEVA 2.1 shows a 20% improvement over FEVA 2.0 in predictive performance. It’s important to remember, however, that the FEVA 2.1 report is a “screening” tool that tells you when a more definitive diagnostic borescope inspection is called for. An analogy in medicine might be the PSA test for prostate cancer. The PSA cannot diagnose prostate cancer, but it can tell you when a definitive test such as a biopsy is advisable.

Math geeks make up a large proportion of the Savvy team! FEVA 2.1 uses a supervised machine learning model, meaning that that model is “trained” using data from flights of aircraft with known valve condition. It is a model that classifies valves into one of three risk categories: below average risk of failure, average risk, and higher than average risk. There are many types of classification models. The one currently used for FEVA 2.1 is a Random Forests ensemble model. We believe this is the most appropriate model for our particular application and the dataset we have at the present time.

FEVA 2.1 reports

If you are a SavvyMx, SavvyQA or SavvyAnalysis Pro client we will email you a FEVA 2.1 report periodically. The frequency will depend on the amount of engine monitor data you upload. The more data you upload, the more frequently you will receive reports.

Valves that fall into the “Average” risk category are predicted to have a probability of failure roughly equal to that of the whole population of valves we examined. Kind of “middle of the pack”. Valves in the “Higher than average” risk category are predicted to have about four times the failure risk of the average valve. In the “Lower than average” risk category, the valve is predicted to have a risk of failure about half of the average valve.

No, not on its own. You should be more focused on the risk category of each cylinder.

If a cylinder falls into the “higher than average” risk category, we believe it has about four times the probability of being in failure than the average cylinder. Does this mean you have a failing valve? Not necessarily. The likelihood of a failing valve is pretty small to begin with. In our test set of data (where we knew the actual condition of every valve) about 4.5% of the valves were in failure. And we believe our test set is biased toward a higher failure rate than the general population because we often learn the actual condition of a valve when someone has a problem and tells us. They tend not to tell us so much when the valve is fine! We think (guessing here) that the real rate of failing valves is around 2-3%.

So we predict the probability of failure at about 8-12% for a cylinder that falls into the “higher than average” risk category.

When should you borescope the “higher than average risk” cylinder? Immediately? The next annual? There are a couple of factors to consider:

  • Does the engine have four cylinders or six? If a valve fails completely in flight, the cylinder will lose all the power from that cylinder. Losing one of four cylinders will have a much more significant impact on performance than losing one of six cylinders. The effect is even worse than it sounds. The climb performance of an aircraft is proportional to the excess power over the minimum power needed to fly straight and level. So losing 25% of your rated power in a four cylinder aircraft will mean losing a lot more than 25% of your climb performance. That could be critical on a short field or high altitude takeoff.
  • When was the cylinder last borescoped and what was the condition at that time? If it hasn’t been a lot of hours since the last borescope inspection, and the valve looked healthy at that time, the chances are higher that your valve is still good. If it was marginal at the last inspection or if it’s been a long time since it was last borescoped, we suggest borescoping as soon as practicable.

If we had to come up with a single rule of thumb for “higher than average risk” cylinders, we’d say borescope no later than your next oil change. The cowling is probably off then anyway, and it’s simple to pull the spark plug and stick the scope in the hole. If you do your own oil changes, so much the better. You can buy a very decent borescope for under $200, so you should probably have one in your toolkit.

If your cylinder is not in the “higher than average risk” category, we suggest you follow our usual advice: have all of your cylinders borescoped at every annual inspection. Even a cylinder in the “lower than average risk” category is not guaranteed to be healthy, it’s just much more likely to be healthy.

Whenever you decide to borescope, whether you do it now or you wait until the next oil change, please share the photos with us — whether the valve is normal or burned.  That’s the only way we can improve our model.

No. FEVA 2.1 does a good job at predicting whether a cylinder’s probability of valve failure is average, above average or below average. However, it is not yet such a precise tool that one can meaningfully compare small differences in report results from different flights. The results will vary from flight to flight, and this is expected.

Digital engine monitors and FEVA 2.1

FEVA 2.1 uses data from all of your recent flight history. We encourage you to upload all of your engine monitor data, and to do so regularly. That way your FEVA 2.1 reports are based on as large, and recent, a data set as possible. It also assists our analysts and account managers in serving you better. There is no limit on the amount of data you can upload and archive on our system.

A one-second sample interval is best because some of the measures used in FEVA 2.1 need high data granularity for accuracy. If your equipment doesn’t support a sample rate as short as one second, program it to the shortest sample interval it does support (e.g., two seconds).

Very possibly. As of now we don’t have an automated way to flag a failing engine monitor probe, although such capability is on the SavvyAI team’s drawing board. If you suspect a bad probe, you should seek advice from a Savvy analyst or account manager.

FEVA 2.1 can only use flights for which the algorithm can identify a cruise segment. The algorithm has an easier time identifying a cruise segment if the data contains altitude information or outside air temperature. It can work with less data, but very basic monitors which record only EGT and CHT are not supported by FEVA 2.1. Such basic monitors will still be supported by FEVA 1.0.