Posts about: "Air Worthiness Directives" [Posts: 40 Pages: 2]

WillowRun 6-3
2025-06-18T15:45:00
permalink
Post: 11905345
Originally Posted by The Brigadier
A case in point is the 2013 Boeing 787 battery fires. After two thermal runaways in 52 000 flight-hours, the root cause was still unknown; however, the FAA nevertheless issued Emergency AD 2013-02-51 grounding every 787 until a modification was available. IMHO a risk where the outcome is catastrophic, even very low probability, would trigger the FAA to issue an Emergency Airworthiness Directive as per their policy.

As I said in a previous post, every day that passes without a EAD suggest the cause was was specific to that aircraft (fuel contamination, maintenance failure, crew error - pick you own theory)
Additionally, the lithium-ion batteries in the 787 design required compliance with "Special Conditions" in order to gain type certification. The addition of the Special Conditions at least strongly suggests that FAA's certification process brain-trust had maintained some vigilance with regard to the batteries once the type started operating, i.e., given their newness.

At this time, the fault or flaw at the root of this accident and how that fault or flaw worked through aircraft components and systems has not been identified publicly. It might not even have been identified .... I'm not convinced that separating "what" from "how and why" as a semantics exercise is enough to conclude the Annex 13 officials know the root cause yet. Maybe they do, maybe not.

And if they do not, then the contrast with the situation where FAA was, it seemed, keeping a close eye on battery issues in particular, is relevant.


3 users liked this post.

Seamless
2025-06-19T14:08:00
permalink
Post: 11906053
I have read most of the thread (old and new). As a lawyer working in forensic investigations, I am constantly involved in problem-solving. My field of work also includes complex investigations related to insolvencies, which almost always require an analysis of the causes behind a specific, established outcome. In doing so, I naturally also have to deal with probabilities. However, it often turns out that the most likely or plausible explanation does not reflect what actually happened.

Many of the considerations I’ve read fail because the simultaneous failure of both engines is extremely unlikely, leading to a constant search for higher-order causes. It was suggested that an incorrect altitude setting led to an early thrust reduction. However, this would not explain the deployment of the RAT (Ram Air Turbine), especially since the thrust could have been readjusted. FADEC and TCAM are highly redundant systems, and TCAM failure is unlikely due to WOW (Weight on Wheels) logic, making a simultaneous engine failure after VR equally improbable.

With that said, and with regard to my question concerning the AD that relates to the fuel control switches (FCS), my thought—and it was nothing more than that—was that their activation becomes more probable if it can occur accidentally. That’s how I came across SAIB: NM-18-33.

Another user then brought up an iPhone. That notion would, of course, be dramatic—but how unlikely is it really that after approximately 10,000 actuations between December 2013 and June 2025, the two FCS no longer lock perfectly? Considering all of this, I find it quite conceivable that the A/T slightly reduced thrust in the first seconds after VR (e.g., if an incorrect target altitude had been entered) and that an object lying between the thrust levers and the FCS could have pushed the FCS into the “Off” position. Due to the buttons on top of the switches, which provide some resistance, it’s even possible that the object both pulled and pushed them.

But all of this is speculation. The investigation report will bring clarity.

Even if my theory is not confirmed, I still believe that the positioning and mechanism of the FCS are suboptimal. Switches of such critical importance should be better protected, and movements in the area in front of the switches (like reducing thrust) should not follow the same direction as shutting off the fuel supply. A different switching direction alone would provide more safety—especially considering that the FCS are protected laterally by metal plates.

5 users liked this post.

DTA
2025-06-19T14:36:00
permalink
Post: 11906073
Originally Posted by Seamless
I have read most of the thread (old and new). As a lawyer working in forensic investigations, I am constantly involved in problem-solving. My field of work also includes complex investigations related to insolvencies, which almost always require an analysis of the causes behind a specific, established outcome. In doing so, I naturally also have to deal with probabilities. However, it often turns out that the most likely or plausible explanation does not reflect what actually happened.

Many of the considerations I\x92ve read fail because the simultaneous failure of both engines is extremely unlikely, leading to a constant search for higher-order causes. It was suggested that an incorrect altitude setting led to an early thrust reduction. However, this would not explain the deployment of the RAT (Ram Air Turbine), especially since the thrust could have been readjusted. FADEC and TCAM are highly redundant systems, and TCAM failure is unlikely due to WOW (Weight on Wheels) logic, making a simultaneous engine failure after VR equally improbable.

With that said, and with regard to my question concerning the AD that relates to the fuel control switches (FCS), my thought\x97and it was nothing more than that\x97was that their activation becomes more probable if it can occur accidentally. That\x92s how I came across SAIB: NM-18-33.

Another user then brought up an iPhone. That notion would, of course, be dramatic\x97but how unlikely is it really that after approximately 10,000 actuations between December 2013 and June 2025, the two FCS no longer lock perfectly? Considering all of this, I find it quite conceivable that the A/T slightly reduced thrust in the first seconds after VR (e.g., if an incorrect target altitude had been entered) and that an object lying between the thrust levers and the FCS could have pushed the FCS into the \x93Off\x94 position. Due to the buttons on top of the switches, which provide some resistance, it\x92s even possible that the object both pulled and pushed them.

But all of this is speculation. The investigation report will bring clarity.

Even if my theory is not confirmed, I still believe that the positioning and mechanism of the FCS are suboptimal. Switches of such critical importance should be better protected, and movements in the area in front of the switches (like reducing thrust) should not follow the same direction as shutting off the fuel supply. A different switching direction alone would provide more safety\x97especially considering that the FCS are protected laterally by metal plates.
It is probable that the switches are becoming easier to move across the gate after 10,000 operations. Something falling on them would be a possibility to cause that. And there is certainly an argument to be had whether down=on is a safer way for them to operate.

6 users liked this post.

skwdenyer
2025-06-19T16:12:00
permalink
Post: 11906161
Originally Posted by StudentInDebt
this isn\x92t the type of switch fitted to the 787 as a fuel control switch, totally irrelevant but has generated yet more nonsense. The switches are spring loaded (or so it feels) in addition to having a massive block to prevent inadvertent operation in either direction. Anyone suggesting they could be accidentally \x93knocked off\x94 is so clueless about their operation it\x92s actually painful to rebut
The type of switch being discussed is the specific type reported as being liable to problems. The SAIB is here https://ad.easa.europa.eu/blob/NM-18...SIB_NM-18-33_1 and specifies a part number for the B788 as 4TL837-3D

That's a TL series switch with 4 poles (the "4" in "4TL"), a "type D" lock (meaning locked out of centre position per the Honeywell data sheet - the "D" in "3D." This is a photo of one:


You can find the manufacturer's datasheet here: https://octopart.com/datasheet/4tl83...ywell-25749542

Problems with critical switches aren't new on 787-8s. For instance, in addition to the SAIB above, there's this AD: https://www.federalregister.gov/docu...pany-airplanes

Looking at the photo above, it isn't just wear that's potentially an issue, but foreign object impingement. There don't appear to be gaitors fitted to these switches in the 788, so the locking mechanisms are potentially susceptible to a build-up of material if not kept clean. There are a range of other failure modes possible, whilst the SAIB specifically describes found-in-the-field problems with these switches.

Yes, they're chunky, and very positive when new. That doesn't mean they shouldn't be discussed.

7 users liked this post.

CloudChasing
2025-06-19T18:05:00
permalink
Post: 11906239
Fuel valves and TCMA software updates?

Originally Posted by tdracer
Commanded engine cutoff - the aisle stand fuel switch sends electrical signals to the spar valve and the "High Pressure Shutoff Valve" (HPSOV) in the Fuel Metering Unit, commanding them to open/close using aircraft power. The HPSOV is solenoid controlled, and near instantaneous. The solenoid is of a 'locking' type that needs to be powered both ways (for obvious reasons, you wouldn't want a loss of electrical power to shut down the engine). The fire handle does the same thing, via different electrical paths (i.e. separate wiring).

As I've noted previously, a complete loss of aircraft electrical power would not cause the engines to flameout (or even lose meaningful thrust) during takeoff. In the takeoff altitude envelope, 'suction feed' (I think Airbus calls it 'gravity feed') is more than sufficient to supply the engine driven fuel pumps. It's only when you get up to ~20k ft. that suction feed can become an issue - and this event happened near sea level.

Not matter what's happening on the aircraft side - pushing the thrust levers to the forward stop will give you (at least) rated takeoff power since the only thing required from the aircraft is fuel and thrust lever position (and the thrust lever position resolver is powered by the FADEC).

The TCMA logic is designed and scrubbed so as to be quite robust - flight test data of the engine response to throttle slams is reviewed to insure there is adequate margin between the TCMA limits and the actual engine responses to prevent improper TCMA activation. Again, never say never, but a whole lot would have had to go wrong in the TCMA logic for it to have activated on this flight.

Now, if I assume the speculation that the RAT deployed is correct, I keep coming up with two potential scenarios that could explain what's known regarding this accident:
1) TCMA activation shutdown the engines
or
2) The fuel cutoff switches were activated.
I literally can come up with no other plausible scenarios.

In all due respect to all the pilots on this forum, I really hope it wasn't TCMA. It wouldn't be the first time a mandated 'safety system' has caused an accident (it wouldn't just be Boeing and GE - TCMA was forced by the FAA and EASA to prevent a scenario that had never caused a fatal accident) - and there would be a lot embarrassing questions for all involved. But I personally know many of the people who created, validated, and certified the GEnx-1B TCMA logic - and can't imagine what they would be going through if they missed something (coincidentally, one of them was at my birthday party last weekend and inevitably we ended up talking about what we used to do at Boeing (he's also retired)). Worse, similar TCMA logic is on the GEnx-2B (747-8) - which I was personally responsible for certifying - as well as the GE90-115B and the 737 MAX Leap engine - the consequences of that logic causing this accident would be massive.
I\x92m sure this is wrong; was looking for confirmation. I read somewhere that the 787 keeps the fuel valve open by an electric driven actuator, and closes it by spring force.

I seem to remember Fred Dibner talking about how railway cars brake by draining the piston not by pressurising it, so trains will stop when supply lines break.

The electrical system updates to 787s for ADs and SBs - do any of these include software updates? For example the integer overflow causing GCU failsafe rectified under AD 2018-20-15. If so, who is writing and implementing these software updates? The original engineers? Their apprentices who had years long handovers? Or have they been outsourced and offshored? When these updates occur, does the entire system get tested and ratified or just the bit the bug fix is meant to fix? Because I\x92ve seen new bugs introduced by bug fixes in areas seemingly nothing to do with the original problem.

skwdenyer
2025-06-19T19:18:00
permalink
Post: 11906289
Originally Posted by galaxy flyer
In the history of jet transport aviation, both ETOPS and non-ETOPS operations, exactly how many simultaneous dual engine failures have there been, excluding pilot causal ones? I\x92d venture it\x92s zero. Even the old DC-9/Boeing 727 era had none. ETOPS is 40 years on and zero cases, to my knowledge. Modern twins are systematically divided into two separate and independent planes. My bet is all these neat theories based on arcane questions will boil down to some human causal event, excluding Boeing. They might contributory, as in the Delta 767 where the switch design contributed to pilot misaction, but design fault, vanishingly improbable.
Dual engine failures? Or uncommanded dual engine shutdowns?

ANA NH-985, a 787-8, suffered dual uncommanded engine shutdown just after air-ground transition. That was a TCMS "feature."

Baltic BT-139 likewise, resulted in an FAA AD to upgrade FADEC software on a whole bunch of P&W engines.

It isn't unheard of. It may not have been seen at rotation before.

1 user liked this post.

Captain Fishy
2025-06-19T20:52:00
permalink
Post: 11906364
Originally Posted by skwdenyer
The type of switch being discussed is the specific type reported as being liable to problems. The SAIB is here https://ad.easa.europa.eu/blob/NM-18...SIB_NM-18-33_1 and specifies a part number for the B788 as 4TL837-3D

That's a TL series switch with 4 poles (the "4" in "4TL"), a "type D" lock (meaning locked out of centre position per the Honeywell data sheet - the "D" in "3D." This is a photo of one:


You can find the manufacturer's datasheet here: https://octopart.com/datasheet/4tl83...ywell-25749542

Problems with critical switches aren't new on 787-8s. For instance, in addition to the SAIB above, there's this AD: https://www.federalregister.gov/docu...pany-airplanes

Looking at the photo above, it isn't just wear that's potentially an issue, but foreign object impingement. There don't appear to be gaitors fitted to these switches in the 788, so the locking mechanisms are potentially susceptible to a build-up of material if not kept clean. There are a range of other failure modes possible, whilst the SAIB specifically describes found-in-the-field problems with these switches.

Yes, they're chunky, and very positive when new. That doesn't mean they shouldn't be discussed.
This switch thing is a nothing burger. If you\x92ve ever operated these switches you\x92d know how they feel. They require a very distinct pull and are most definitely either on or off. There is no impossibly balanced in between position.

8 users liked this post.

Seamless
2025-06-19T21:08:00
permalink
Post: 11906375
Originally Posted by Captain Fishy
This switch thing is a nothing burger. If you\x92ve ever operated these switches you\x92d know how they feel. They require a very distinct pull and are most definitely either on or off. There is no impossibly balanced in between position.
So why the AD back in 2018?
skwdenyer
2025-06-20T06:18:00
permalink
Post: 11906620
Originally Posted by ignorantAndroid
TCMA is simply a bit of software in the FADECs, so it has the same separation as everything else. There's no inter-engine interaction when it comes to TCMA.
There is no inter-engine interaction. But if both FADECs are running the same software then they will potentially behave identically to some bug or external input / condition. That's why software redundancy isn't always redundancy.

Originally Posted by ignorantAndroid
This is technically possible, but of course the FADECs would have the ability to shut down the engines anyway, even if TCMA didn't exist. If there's a software bug, it could involve TCMA but it could easily be somewhere else.
I agree.

Originally Posted by ignorantAndroid
TCMA can't be disabled electrically. It's just software, and all of the hardware involved serves other functions which are still needed while in the air. For example, the FADECs would command the HPSOV closed in case of N2 overspeed. That would have the exact same effect as TCMA.
Thank you. So saying "TCMA would only trigger if WoW and RA have failed somehow" is incorrect? Those are simply inputs to software, which might itself fail badly for other reasons. See the FAA AD to replace FADEC software on certain P&W engines following a previous uncommanded dual engine shutdown.

4 users liked this post.

JustusW
2025-06-20T17:45:00
permalink
Post: 11907155
Originally Posted by Kraftstoffvondesibel
1/Would the recorders lose access to aircraft data streams when engine power is lost, at least temporarely making the cockpit area mic recorded by battery power on the front recorder the only source of information ?
2/The recorders only draw 20W, why is it the front with reserves only for 10 minutes? Can you even buy a battery that small giving 28VDC? Why is such a limited solution selected?
( For reference, This battery gives about 9 times that: <url removed> )
In reverse order, and the first one being very speculative: The type of battery will likely be highly specific for the usecase, here rugged before anything else. Likely specialized chemistry or one of those hybrid solid state ones. Commonly they trade capacity for other features.

Regarding the recording feature, there's three types of microphone commonly used nowadays: Condenser and Ribbon type are somewhat fragile and require power to record audio while Dynamic type is basically a reverse speaker and is considered rugged. There's an off chance that a Piezzo microphone would be used here as they are basically indestructible but usually reserved for recording while in contact with a large sound transducer. My guess based on that is that we're looking at a dynamic microphone with a run of the mill preamp.
Depending on the actual electric setup this would yield a handful of different possible installations:
1) The "Cockpit Area Microphone" (hereby christened CAM because I like abbreviations) is a self contained unit consisting of a Microphone, a preamp and AD converter. This would mean while provided power the digital recording could be passed to either EAFR.
2) The CAM is a self contained unit consisting of a Microphone and a preamp. This would mean while provided power it could send an analog audio signal to the forward EAFR no problem, but would potentially struggle generating enough of a signal to be picked up by the rear EAFR.
3) The CAM is just a Microphone. This would mean it requires either no or very little power (even Condenser Mics usually require only Milliwatts) but the signal would be very hard to send over long distances and would require the EAFR to have a preamp.

In general it is audio engineering 101 to place a preamp as close to the source as possible to avoid noise. Thus I would rule out 3. It has both ups and downs to convert the analog signal to a digital signal, and there is a possibility they'd do both. In either case I am confused from an audio engineering standpoint why the rear EAFR would not pickup audio from the CAM if the forward EAFR does. Unless the rear EAFR is fed (audio) data only via BUS, which would be an interesting choice.
Also keep in mind that historically the CVR was also located in the tail section and very much received an analog signal over the entire distance. There's really no technical reason this wouldn't be possible, I routinely use far longer cables when running audio signals at concerts and those can't use compression because it would dumpster sound quality.

So, yeah, I don't understand why there would be a mismatch between the recordings of either EAFR, unless there was something else preventing all signal transmission towards the rear EAFR. The CVR in the rear has been a thing for 80 years now.

Regards,
Justus

2 users liked this post.

Kraftstoffvondesibel
2025-06-20T19:10:00
permalink
Post: 11907217
Originally Posted by JustusW
In reverse order, and the first one being very speculative: The type of battery will likely be highly specific for the usecase, here rugged before anything else. Likely specialized chemistry or one of those hybrid solid state ones. Commonly they trade capacity for other features.

Regarding the recording feature, there's three types of microphone commonly used nowadays: Condenser and Ribbon type are somewhat fragile and require power to record audio while Dynamic type is basically a reverse speaker and is considered rugged. There's an off chance that a Piezzo microphone would be used here as they are basically indestructible but usually reserved for recording while in contact with a large sound transducer. My guess based on that is that we're looking at a dynamic microphone with a run of the mill preamp.
Depending on the actual electric setup this would yield a handful of different possible installations:
1) The "Cockpit Area Microphone" (hereby christened CAM because I like abbreviations) is a self contained unit consisting of a Microphone, a preamp and AD converter. This would mean while provided power the digital recording could be passed to either EAFR.
2) The CAM is a self contained unit consisting of a Microphone and a preamp. This would mean while provided power it could send an analog audio signal to the forward EAFR no problem, but would potentially struggle generating enough of a signal to be picked up by the rear EAFR.
3) The CAM is just a Microphone. This would mean it requires either no or very little power (even Condenser Mics usually require only Milliwatts) but the signal would be very hard to send over long distances and would require the EAFR to have a preamp.

In general it is audio engineering 101 to place a preamp as close to the source as possible to avoid noise. Thus I would rule out 3. It has both ups and downs to convert the analog signal to a digital signal, and there is a possibility they'd do both. In either case I am confused from an audio engineering standpoint why the rear EAFR would not pickup audio from the CAM if the forward EAFR does. Unless the rear EAFR is fed (audio) data only via BUS, which would be an interesting choice.
Also keep in mind that historically the CVR was also located in the tail section and very much received an analog signal over the entire distance. There's really no technical reason this wouldn't be possible, I routinely use far longer cables when running audio signals at concerts and those can't use compression because it would dumpster sound quality.

So, yeah, I don't understand why there would be a mismatch between the recordings of either EAFR, unless there was something else preventing all signal transmission towards the rear EAFR. The CVR in the rear has been a thing for 80 years now.

Regards,
Justus
The recorder data sheet specifies it is an analog input for the area mic, and that it and its pre-amp are powered by the recorder.
It is likely a mems-type microphone, moving coil, ribbon or traditional condenser microphones aren’t really used outside the stage or vintage recording studios these days. But something along these lines: https://pdf.aeroexpo.online/pdf/l3-t...html#open64169

Note the dual analog and arinc digital outputs.

One reason for not doing an analog line all the way to the tail would be weight, as you mention, quality or noise wouldn’t be an issue due common mode rejection.

Last edited by T28B; 20th Jun 2025 at 19:11 . Reason: punctuation and grammar assist

3 users liked this post.

JustusW
2025-06-21T17:04:00
permalink
Post: 11907907
Originally Posted by Lead Balloon
And I reiterate the point that the TCMA is "just software". I haven't seen anyone dispute the suggestion that the thing 'common' to all 4 channels of the TCMA is 4 copies of software.
This has been setting a bit wrong with me for the entirety of the thread. When people say "software" they have an image in their head, and in this case the image is rather unhelpful if not outright wrong.
When we say "Software" we mean things like this:
if ( happy == true ) {
print("I'm happy!"
}
This is the type of software we are usually subject to in our everyday lives basically everywhere. Your phone, your fridge, your oven, your water heater, your car, etc. pp. ad nauseam.

In case of the Safran FADEC 3 this is not actually what we're dealing with. It uses something called an FPGA (Field Programmable Gate Array) which is a very different beast to what we are used to dealing with.

In a normal computer we simply translate code like above into something less human readable but instead interpretable by a microchip or the like. This "machine code" is then run sequentially and does whatever it is meant to do (hopefully). If we were to expand our little happy check above with another condition then our computer would sequentially check these conditions. This opens up a lot of hoopla about timing, especially when interacting with the real world.

Let's say you want to track N1 and N2 in an engine. You'd have to make a measurement, that value would have to go through an AD converter, which writes it to some (likely volatile) storage, where it is then accessed by some sort of (C)PU, transferred to yet another piece of volatile memory and then used in the computation. This takes time because you can't do those things within the same clock cycle.

Unsurprisingly this is rather inconvenient when dealing with the real world and especially when dealing with volatile physical processes that need monitoring. Like a modern turbine engine.

Enter the FPGA.

While it is programmable what that actually means is that (at a very high level) you can build a thing called a truth table, that means a definitive mathematical mapping of input states to output states. Unlike our sequential CPU driven system an FPGA will be able to perform its entire logic every time it is asked to do so. We don't have to wait for our happy check to perform any other check.

This is very useful for our Turbine Engine, because now we can verify that N2 is smaller than X without delaying our check that the Throttle Control Lever setting is within acceptable distance to N1 and N2, while also creating the appropriate output for all the different bypasses and bleed valves in a modern engine, and so on. The Safran FADEC 3 does this "up to 70 times per second" as per the vendor.

In order to be manageable the actual FADEC consists of multiple different pieces, many of which are FPGAs. At least two are used for so called "input conditioning". This is where we deal with converting messy real world values from sensors subject to real physics into nice and clean numbers. Here our values from the Throttle Control Levers come in, the signal from our N1 and N2 sensors, WOW signal, and on, and on, and on.

This is then changed into logic level signals provided to a different set of FPGAs. Lacking an actual schematic I suspect that this is what sometimes is referred to as "channels". The channel could consist of both a signal conditioning stage and then a logic stage or (more likely) redundant signal conditioning stages feed separately into separate FPGA "channels" are evaluated and then the end result is likely put into yet another FPGA for control output generation.

Why is this important?

Because it is basically impossible for a "bug" to exist in this system. These systems are the epitome of a dumb computer. They do _precisely_ what they are meant to do. The TCMA is simply a set of conditions evaluated just like any other condition in those FPGAs. If there is an "error" then it is in the requirements that lead to the truth table used by these devices but never in the truth table itself. In fact these types of computation device are used precisely because of that very property.

So when people say "the FADEC software" (ie TCMA) has "failed" or "has a bug" what they're really saying is:
The conditions that were experienced in the real world by the system were incorrectly assessed in the system requirements and lead to an undesired output state when taking into account the real world result we would have preferred.
A bit of a mouth full, granted, but an important mouth full. This simply wouldn't be someone missing a semicolon somewhere in a thousand lines of code.

Regards,
Justus

Last edited by T28B; 21st Jun 2025 at 17:25 . Reason: Formatting making this easier to read for the layman. Nice post.

26 users liked this post.

skwdenyer
2025-06-21T18:31:00
permalink
Post: 11907969
Originally Posted by JustusW
This has been setting a bit wrong with me for the entirety of the thread. When people say "software" they have an image in their head, and in this case the image is rather unhelpful if not outright wrong.
When we say "Software" we mean things like this:

This is the type of software we are usually subject to in our everyday lives basically everywhere. Your phone, your fridge, your oven, your water heater, your car, etc. pp. ad nauseam.

In case of the Safran FADEC 3 this is not actually what we're dealing with. It uses something called an FPGA (Field Programmable Gate Array) which is a very different beast to what we are used to dealing with.

In a normal computer we simply translate code like above into something less human readable but instead interpretable by a microchip or the like. This "machine code" is then run sequentially and does whatever it is meant to do (hopefully). If we were to expand our little happy check above with another condition then our computer would sequentially check these conditions. This opens up a lot of hoopla about timing, especially when interacting with the real world.

Let's say you want to track N1 and N2 in an engine. You'd have to make a measurement, that value would have to go through an AD converter, which writes it to some (likely volatile) storage, where it is then accessed by some sort of (C)PU, transferred to yet another piece of volatile memory and then used in the computation. This takes time because you can't do those things within the same clock cycle.

Unsurprisingly this is rather inconvenient when dealing with the real world and especially when dealing with volatile physical processes that need monitoring. Like a modern turbine engine.

Enter the FPGA.

While it is programmable what that actually means is that (at a very high level) you can build a thing called a truth table, that means a definitive mathematical mapping of input states to output states. Unlike our sequential CPU driven system an FPGA will be able to perform its entire logic every time it is asked to do so. We don't have to wait for our happy check to perform any other check.

This is very useful for our Turbine Engine, because now we can verify that N2 is smaller than X without delaying our check that the Throttle Control Lever setting is within acceptable distance to N1 and N2, while also creating the appropriate output for all the different bypasses and bleed valves in a modern engine, and so on. The Safran FADEC 3 does this "up to 70 times per second" as per the vendor.

In order to be manageable the actual FADEC consists of multiple different pieces, many of which are FPGAs. At least two are used for so called "input conditioning". This is where we deal with converting messy real world values from sensors subject to real physics into nice and clean numbers. Here our values from the Throttle Control Levers come in, the signal from our N1 and N2 sensors, WOW signal, and on, and on, and on.

This is then changed into logic level signals provided to a different set of FPGAs. Lacking an actual schematic I suspect that this is what sometimes is referred to as "channels". The channel could consist of both a signal conditioning stage and then a logic stage or (more likely) redundant signal conditioning stages feed separately into separate FPGA "channels" are evaluated and then the end result is likely put into yet another FPGA for control output generation.

Why is this important?

Because it is basically impossible for a "bug" to exist in this system. These systems are the epitome of a dumb computer. They do _precisely_ what they are meant to do. The TCMA is simply a set of conditions evaluated just like any other condition in those FPGAs. If there is an "error" then it is in the requirements that lead to the truth table used by these devices but never in the truth table itself. In fact these types of computation device are used precisely because of that very property.

So when people say "the FADEC software" (ie TCMA) has "failed" or "has a bug" what they're really saying is:
The conditions that were experienced in the real world by the system were incorrectly assessed in the system requirements and lead to an undesired output state when taking into account the real world result we would have preferred.
A bit of a mouth full, granted, but an important mouth full. This simply wouldn't be someone missing a semicolon somewhere in a thousand lines of code.

Regards,
Justus
Whilst you\x92re right to highlight the type of software in a FADEC, it should also be recalled that after the dual uncommanded engine shutdown event on the Baltic CS300, the FAA mandated updated FADEC software for all of a family of P&W engines. FADEC s/w can contain what we\x92d call bugs (whether they\x92re mistakes of coding or mistakes of logic in the design assumptions).
Furr
2025-06-21T18:37:00
permalink
Post: 11907975
Originally Posted by JustusW
This has been setting a bit wrong with me for the entirety of the thread. When people say "software" they have an image in their head, and in this case the image is rather unhelpful if not outright wrong.
When we say "Software" we mean things like this:

This is the type of software we are usually subject to in our everyday lives basically everywhere. Your phone, your fridge, your oven, your water heater, your car, etc. pp. ad nauseam.

In case of the Safran FADEC 3 this is not actually what we're dealing with. It uses something called an FPGA (Field Programmable Gate Array) which is a very different beast to what we are used to dealing with.

In a normal computer we simply translate code like above into something less human readable but instead interpretable by a microchip or the like. This "machine code" is then run sequentially and does whatever it is meant to do (hopefully). If we were to expand our little happy check above with another condition then our computer would sequentially check these conditions. This opens up a lot of hoopla about timing, especially when interacting with the real world.

Let's say you want to track N1 and N2 in an engine. You'd have to make a measurement, that value would have to go through an AD converter, which writes it to some (likely volatile) storage, where it is then accessed by some sort of (C)PU, transferred to yet another piece of volatile memory and then used in the computation. This takes time because you can't do those things within the same clock cycle.

Unsurprisingly this is rather inconvenient when dealing with the real world and especially when dealing with volatile physical processes that need monitoring. Like a modern turbine engine.

Enter the FPGA.

While it is programmable what that actually means is that (at a very high level) you can build a thing called a truth table, that means a definitive mathematical mapping of input states to output states. Unlike our sequential CPU driven system an FPGA will be able to perform its entire logic every time it is asked to do so. We don't have to wait for our happy check to perform any other check.

This is very useful for our Turbine Engine, because now we can verify that N2 is smaller than X without delaying our check that the Throttle Control Lever setting is within acceptable distance to N1 and N2, while also creating the appropriate output for all the different bypasses and bleed valves in a modern engine, and so on. The Safran FADEC 3 does this "up to 70 times per second" as per the vendor.

In order to be manageable the actual FADEC consists of multiple different pieces, many of which are FPGAs. At least two are used for so called "input conditioning". This is where we deal with converting messy real world values from sensors subject to real physics into nice and clean numbers. Here our values from the Throttle Control Levers come in, the signal from our N1 and N2 sensors, WOW signal, and on, and on, and on.

This is then changed into logic level signals provided to a different set of FPGAs. Lacking an actual schematic I suspect that this is what sometimes is referred to as "channels". The channel could consist of both a signal conditioning stage and then a logic stage or (more likely) redundant signal conditioning stages feed separately into separate FPGA "channels" are evaluated and then the end result is likely put into yet another FPGA for control output generation.

Why is this important?

Because it is basically impossible for a "bug" to exist in this system. These systems are the epitome of a dumb computer. They do _precisely_ what they are meant to do. The TCMA is simply a set of conditions evaluated just like any other condition in those FPGAs. If there is an "error" then it is in the requirements that lead to the truth table used by these devices but never in the truth table itself. In fact these types of computation device are used precisely because of that very property.

So when people say "the FADEC software" (ie TCMA) has "failed" or "has a bug" what they're really saying is:
The conditions that were experienced in the real world by the system were incorrectly assessed in the system requirements and lead to an undesired output state when taking into account the real world result we would have preferred.
A bit of a mouth full, granted, but an important mouth full. This simply wouldn't be someone missing a semicolon somewhere in a thousand lines of code.

Regards,
Justus
"it is basically impossible for a "bug" to exist in this system." This is not my experience.There are a lot of bugs in hardware and FPGAs. The guys doing this are real experts and bugs are less common than in software. But they do exist and they can be difficult to reproduce and subtle to diagnose. For someone new to FPGA it can be very intimidating because everything is parallel. It requires a different way of thinking. You can execute 1000 instructions simultaneously.Then various tricks to speed things up or reduce glitches add complexity. For example you may want to count in gray code or one hot instead of binary. Writing bug free FPGA code is very challenging. We cannot assume the FPGA code is free of bugs.

4 users liked this post.

Feathers McGraw
2025-06-21T19:24:00
permalink
Post: 11907998
Originally Posted by JustusW
This has been setting a bit wrong with me for the entirety of the thread. When people say "software" they have an image in their head, and in this case the image is rather unhelpful if not outright wrong.
When we say "Software" we mean things like this:

This is the type of software we are usually subject to in our everyday lives basically everywhere. Your phone, your fridge, your oven, your water heater, your car, etc. pp. ad nauseam.

In case of the Safran FADEC 3 this is not actually what we're dealing with. It uses something called an FPGA (Field Programmable Gate Array) which is a very different beast to what we are used to dealing with.

In a normal computer we simply translate code like above into something less human readable but instead interpretable by a microchip or the like. This "machine code" is then run sequentially and does whatever it is meant to do (hopefully). If we were to expand our little happy check above with another condition then our computer would sequentially check these conditions. This opens up a lot of hoopla about timing, especially when interacting with the real world.

Let's say you want to track N1 and N2 in an engine. You'd have to make a measurement, that value would have to go through an AD converter, which writes it to some (likely volatile) storage, where it is then accessed by some sort of (C)PU, transferred to yet another piece of volatile memory and then used in the computation. This takes time because you can't do those things within the same clock cycle.

Unsurprisingly this is rather inconvenient when dealing with the real world and especially when dealing with volatile physical processes that need monitoring. Like a modern turbine engine.

Enter the FPGA.

While it is programmable what that actually means is that (at a very high level) you can build a thing called a truth table, that means a definitive mathematical mapping of input states to output states. Unlike our sequential CPU driven system an FPGA will be able to perform its entire logic every time it is asked to do so. We don't have to wait for our happy check to perform any other check.

This is very useful for our Turbine Engine, because now we can verify that N2 is smaller than X without delaying our check that the Throttle Control Lever setting is within acceptable distance to N1 and N2, while also creating the appropriate output for all the different bypasses and bleed valves in a modern engine, and so on. The Safran FADEC 3 does this "up to 70 times per second" as per the vendor.

In order to be manageable the actual FADEC consists of multiple different pieces, many of which are FPGAs. At least two are used for so called "input conditioning". This is where we deal with converting messy real world values from sensors subject to real physics into nice and clean numbers. Here our values from the Throttle Control Levers come in, the signal from our N1 and N2 sensors, WOW signal, and on, and on, and on.

This is then changed into logic level signals provided to a different set of FPGAs. Lacking an actual schematic I suspect that this is what sometimes is referred to as "channels". The channel could consist of both a signal conditioning stage and then a logic stage or (more likely) redundant signal conditioning stages feed separately into separate FPGA "channels" are evaluated and then the end result is likely put into yet another FPGA for control output generation.

Why is this important?

Because it is basically impossible for a "bug" to exist in this system. These systems are the epitome of a dumb computer. They do _precisely_ what they are meant to do. The TCMA is simply a set of conditions evaluated just like any other condition in those FPGAs. If there is an "error" then it is in the requirements that lead to the truth table used by these devices but never in the truth table itself. In fact these types of computation device are used precisely because of that very property.

So when people say "the FADEC software" (ie TCMA) has "failed" or "has a bug" what they're really saying is:
The conditions that were experienced in the real world by the system were incorrectly assessed in the system requirements and lead to an undesired output state when taking into account the real world result we would have preferred.
A bit of a mouth full, granted, but an important mouth full. This simply wouldn't be someone missing a semicolon somewhere in a thousand lines of code.

Regards,
Justus
I will just comment here that FPGAs contain many logic blocks which have around them routing channels which allow these logic blocks to be connected up in various ways, some of the logic is asynchronous, some of it is synchronous and thus uses clock signals to advance the state of the logic according to a combination on input signals, asynchronous logic outputs and signals fed back from other logic blocks.

All of this is created by writing software using something like VHDL (VHSIC Hardware Description Language where VHSIC stands for Very High Speed Integrated Circuit) with the output of the compiler for this software being an object file that contains the connection information for the on-FPGA routing. However, this process requires careful simulation, not just of the logical behaviour of the FPGA being designed but also of the ability of the programmable routing on the FPGA to get the required signals to where they are required with the necessary set-up delays so that the next clock edge does not allow a glitch where a signal changes state during the period when it must be static so that it propagates correctly through the logic blocks. With clock signals that form a clock tree across the device the skew on the clock signals must be carefully checked so that the lowest skew requirements are always met, there may be more margin for the less critical parts of the system.

Now, I am sure that the state of the art has moved on a lot since I was heavily involved in this sort of engineering but I can say that when my colleagues were prototyping ASICs (which are application specific devices that are not programmable like an FPGA) using FPGAs because it can be reprogrammed to cure bugs found in the code that created it I am absolutely sure that it is possible for a latent bug to exist in what has been built despite extensive testing being carried out. I well remember sitting down with an FPGA designer and a software engineer and showing them a situation where one of my radio devices was not being commanded to transmit, they refused to believe me until we captured their output signals in the failed state and I was thus able to prove to them that it was their logic and code that was failing and that my radio system was fully working except it couldn't possibly transmit when its enable line was inactive.

I would therefore never state that an FPGA-based FADEC is infallible. The devil will be in the detail of the functions contained within the devices and how they interact with each other and how the original logical functions are described and specified and then coded and checked for logical errors before getting on to the actual physical reality of the FPGA manufacturer's design and development system that bolts on to the back of the source code. If the FPGA devices are being pushed anywhere near the edge of their performance envelope, just like an aircraft things can go wrong. If a design begins with a device that is only just large enough in terms of how many logic blocks and routing channels are available for the logic required it's almost a certainty that development will take it to this area of its performance and a lot of tweaking may be needed which means that even more testing will be needed to be reasonable sure that it does what it is supposed to.

9 users liked this post.

Semreh
2025-06-22T16:37:00
permalink
Post: 11908670
Originally Posted by JustusW
In reverse order, and the first one being very speculative: The type of battery will likely be highly specific for the usecase, here rugged before anything else. Likely specialized chemistry or one of those hybrid solid state ones. Commonly they trade capacity for other features.

Regarding the recording feature, there's three types of microphone commonly used nowadays: Condenser and Ribbon type are somewhat fragile and require power to record audio while Dynamic type is basically a reverse speaker and is considered rugged. There's an off chance that a Piezzo microphone would be used here as they are basically indestructible but usually reserved for recording while in contact with a large sound transducer. My guess based on that is that we're looking at a dynamic microphone with a run of the mill preamp.
Depending on the actual electric setup this would yield a handful of different possible installations:
1) The "Cockpit Area Microphone" (hereby christened CAM because I like abbreviations) is a self contained unit consisting of a Microphone, a preamp and AD converter. This would mean while provided power the digital recording could be passed to either EAFR.
2) The CAM is a self contained unit consisting of a Microphone and a preamp. This would mean while provided power it could send an analog audio signal to the forward EAFR no problem, but would potentially struggle generating enough of a signal to be picked up by the rear EAFR.
3) The CAM is just a Microphone. This would mean it requires either no or very little power (even Condenser Mics usually require only Milliwatts) but the signal would be very hard to send over long distances and would require the EAFR to have a preamp.

In general it is audio engineering 101 to place a preamp as close to the source as possible to avoid noise. Thus I would rule out 3. It has both ups and downs to convert the analog signal to a digital signal, and there is a possibility they'd do both. In either case I am confused from an audio engineering standpoint why the rear EAFR would not pickup audio from the CAM if the forward EAFR does. Unless the rear EAFR is fed (audio) data only via BUS, which would be an interesting choice.
Also keep in mind that historically the CVR was also located in the tail section and very much received an analog signal over the entire distance. There's really no technical reason this wouldn't be possible, I routinely use far longer cables when running audio signals at concerts and those can't use compression because it would dumpster sound quality.

So, yeah, I don't understand why there would be a mismatch between the recordings of either EAFR, unless there was something else preventing all signal transmission towards the rear EAFR. The CVR in the rear has been a thing for 80 years now.

Regards,
Justus
SLF here. Mods - please delete summarily if this does not contribute to the discussion, I have no wish to waste anyones time. No 'AI' was used in the preparation of this post.

My understanding is that, as you say, the CAM has a preamp. That preamp can be powered by the RIPS that accompanies the forward EAFR.
In addition, I believe there is a single analogue connection from the CAM+preamp to the aft EAFR in addition to the analogue connection from the CAM+preamp to the forward EAFR. I believe, but am not sure,that the other flight-deck audio (headsets) is carried digitally over the fibre-optic network to the aft EAFR. The network may or may not be in operation in the event of an electrical failure: I simply don't know.

The publicly available information I can find is not stunningly clear about this.

AEROSAFETY WORLD, January 2008 - https://flightsafety.org/asw/jan08/a...47-48.pdf?dl=1

In the 787, the EAFRs store within their CVR-function memory partitions two hours of data from four audio channels and all data link messages. \x93The CVR function receives audio from three digital audio crew channels provided by the flight deck audio system and one analog audio channel from the cockpit area microphone and preamplifier,\x94 Elliott said.( Jim Elliott, a systems/applications engineer for the manufacturer. )
GE Aviation: Consolidate and increase recording power with the 3254F EAFR. - https://www.geaerospace.com/sites/de...rder-3254F.pdf

The Cockpit Voice Recorder function records the flight deck communications between crew members and also captures the general acoustical sound environment of the flight deck. The CVR function receives three analog audio crew channels provided by the Flight Deck Audio System and one analog audio channel from the cockpit Area Microphone and Preamplifier (AMP). The cockpit area audio and the three audio crew channels are recorded in both the forward and the aft installed EAFR recorders. The CVR recording duration is two hours minimum. Recorded audio can only be downloaded when the EAFR is off the aircraft.
As for power, this NTSB document describes the power set-up for the EAFRs

https://data.ntsb.gov/Docket/Documen...ort-Master.PDF

Two EAFRs are installed on Boeing 787 aircraft, one forward and one aft. The forward and aft recorders are powered by the left and right 28V DC buses respectively. The forward recorder is equipped with a recorder independent power supply (RIPS) to provide backup power to the recorder for approximately 10 minutes once left DC bus power is lost. Both recorders record the same set of flight data independent of each other.
So the forward EAFR is powered from the left 28V DC bus with the possibility of being powered by the RIPS, and the aft EAFR is powered from the right 28 V DC bus.

What I have been unable to determine is whether the right and/or left 28 V DC buses are powered from the main battery in case of failure of the AC power supply. To my untrained eye, it looks like the Captain's flight displays are powered from the main battery in extremis (28 V DC - C1), but that there are various circuit breakers, that could be automated, that may or may not allow or prevent other loads (such as the F/O's flight displays (28 V DC - C2), or the aft EAFR, being supplied by the main battery, (See link to diagram). There could well be very drastic automated load shedding.

https://kb.skyhightex.com/wp-content...l-1024x640.png

If the right 28 V DC bus was unpowered for any period, it follows that the aft EAFR was not recording for that period. This would make the forward EAFR important in case of a power failure that prevented the right 28 V DC bus from providing power.

All the information that is unclear to me will be transparently clear to the crash investigators. But it seems to me that the aft EAFR will not hold data for any period that the right 28 V DC bus is not operating. Whether that applies to this incident is an open question.
MaybeItIs
2025-06-28T03:55:00
permalink
Post: 11912318
Originally Posted by Capn Bloggs
I'm not suggesting he's wrong about the ADs, I'm just saying that water in the avionics bay (which I'm sure has been brought up already) is pure speculation.
...by the FAA

"This AD was prompted by reports of water leakage from the potable water system due to improperly installed waterline couplings, and water leaking into the electronics equipment (EE) bays from above the floor in the main cabin, resulting in water on the equipment in the EE bays"
fdr
2025-06-30T03:37:00
permalink
Post: 11913337
Originally Posted by TURIN
It will come as no surprise to anyone that EE bays are well protected with the sort of things you have described.
The 787 batteries are also in separate EE bays. Main one in the front and the dedicated APU battery in the power electrics bay aft of the landing gear.
They are both contained in fireproof boxes that will vent to atmosphere in the event of a thermal runaway.
I have been working on 787s for over a decade and leaks from gallies and lavs has not once been on my list of snags.
The first B744 flood event in the E&E was 25 years into the operation with one airline, and the other 2 occurred over 30 years into the operation of the type, one was uniquely a cargo loading event, and how the OEM would have guessed that some carrier would enter a wading pool worth of rain water into the E&E, they had and have my sympathy. We took action after that to avoid a repetition, but it wasn't on the radar before it ended up with a bit of excitement for the crew. They happened to have a nice clear evening when the whole cockpit went darl. The B787 has a AD out since 2016 which was added to last year related to unwanted leaks. My own event with a B777 ended up with over 6 cubic meters of ice in the belly of the B773ER, and was only found doing a walk around where there was a number of the belly drains drooling onto the ramp, which happened to be dry. An event external to the system architecture remains a high probability, and as unusual as that may be it is not without precedent, with existing AD's related to such matters extant.

When pax flush clothing and other rubbish down a vacuum toilet system, the potential for stuff to not work as advertised is not zero.

9 users liked this post.

adfad
2025-07-01T12:55:00
permalink
Post: 11914255
Originally Posted by Someone Somewhere
I believe that particular bug is fixed, though it's always possible there's other issues causing a total AC loss.

Not really relevant to what you quoted though, as the scenario in question requires:
  • Engines running on centre tank fuel during takeoff while the aircraft is operating normally
    • We don't know for certain if this is the case. It seems to be but it's not something that happens on other families.
  • Then, total AC failure stopping fuel boost pumps.
  • Engines suction feed from contaminated/full-of-water wing tanks.

The aircraft has two engines and should be able to climb out on one, plus it dropped like a rock . 'Significantly degraded' thrust isn't really compatible with what we saw. You'd also expect the engines to recover pretty quickly as it leveled off.

The limitations at high altitude are primarily air/volatiles degassing out of the fuel. That's not going to be much of an issue at sea level, even if the engines are a bit higher up during rotation.
APU is a nice-to-have; it's on the MEL. If you lose all four generators, it's because of some major carnage in the electrical software/hardware and chances of putting the APU on line even if it's operating are very slim.
As an electronics and software engineer who has read the AD and related materials on the 248 day bug my understanding is that:
  1. The specific 248-day integer overflow was patched, and before the fix was rolled out, the AD required this system to by power cycled every 120 days to prevent overflow
  2. The PCU software still has the functional requirement to be able to command all AC GCUs to enter failsafe mode, this means that while the initial bug was fixed, the ability for this particular software system to command the same result is still a functional part of the architecture - presumably for safety management of the AC system
  3. This was not the first or last "software overflow error" issue in Boeing or even in the 787
Although I'm not qualified in aviation engineering I do believe from an engineering safety standpoint that this architecture creates a rare but entirely feasible scenario in which the aircraft would be without AC power for at least 30 seconds until the APU could restore it.

I do agree that the engine driven pumps should be able to provide fuel alone, the whole point of these pumps is to keep the plane flying within some limitations, high altitude is one of those limitations, I propose that there may be others based on the following:
  • Some more knowledgable people here have proposed or countered vapour lock, fuel contamination and automatic fuel cut-off theories to various degrees - even if these are not enough on their own, loss of electrical during rotation at high temperature could combine with these in a way we have not yet considered
  • Thrust is nonlinear, and while I'm not qualified to say how much loss of fuel flow or loss of thrust would be critical in this scenario we do know that it was a hot takeoff with significant weight and gear remaining down - I know others here have run sims but I don't think anyone has focused on specific thrust / fuel flow params
  • While electric fuel pumps might not be physically necessary for takeoff, my final point is: why are they required for takeoff? Is it not to mitigate cavitation, fuel sloshing at rotation, or any other kind of problem that might be relevant here?
Someone Somewhere
2025-07-01T13:08:00
permalink
Post: 11914265
Originally Posted by adfad
As an electronics and software engineer who has read the AD and related materials on the 248 day bug my understanding is that:
  1. The specific 248-day integer overflow was patched, and before the fix was rolled out, the AD required this system to by power cycled every 120 days to prevent overflow
  2. The PCU software still has the functional requirement to be able to command all AC GCUs to enter failsafe mode, this means that while the initial bug was fixed, the ability for this particular software system to command the same result is still a functional part of the architecture - presumably for safety management of the AC system
  3. This was not the first or last "software overflow error" issue in Boeing or even in the 787
Although I'm not qualified in aviation engineering I do believe from an engineering safety standpoint that this architecture creates a rare but entirely feasible scenario in which the aircraft would be without AC power for at least 30 seconds until the APU could restore it.
Similar failures have happened on 737s/A320s/A330s and others. I'm not denying it's possible. There's a reason it's a certification requirement for the engines not to be dependent on aircraft power. The APU is MELable and battery starts are not extremely reliable.

I do agree that the engine driven pumps should be able to provide fuel alone, the whole point of these pumps is to keep the plane flying within some limitations, high altitude is one of those limitations, I propose that there may be others based on the following:
  • Some more knowledgable people here have proposed or countered vapour lock, fuel contamination and automatic fuel cut-off theories to various degrees - even if these are not enough on their own, loss of electrical during rotation at high temperature could combine with these in a way we have not yet considered
  • Thrust is nonlinear, and while I'm not qualified to say how much loss of fuel flow or loss of thrust would be critical in this scenario we do know that it was a hot takeoff with significant weight and gear remaining down - I know others here have run sims but I don't think anyone has focused on specific thrust / fuel flow params
  • While electric fuel pumps might not be physically necessary for takeoff, my final point is: why are they required for takeoff? Is it not to mitigate cavitation, fuel sloshing at rotation, or any other kind of problem that might be relevant here?
Thrust is non-linear and complex. Reaction engines (i.e. fans, props) are generally most efficient at minimum power - lowest excess velocity. Turbine engines are generally most efficient at high power. These cancel out somewhere in the middle. With two engines at low power, you also don't have the drag from the dead engine or the drag from the rudder countering yaw.

Cavitating destroys pumps rapidly - someone upthread said replacing the fuel pump immediately is SOP if it has suction fed. Expect end of life in tens of hours rather than tens of thousands.

Some aircraft have switched to using jet/venturi pumps powered by returned fuel, like the A220. The electric boost pumps there are mainly for redundancy and are shut down in cruise; only one in each wing tank. Some A320s replace the centre override pumps with venturi transfer pumps.