2025-06-15T00:10:00
permalink Post: 11901981
Rather more importantly, a billion-to-one chance can happen on the first try with equal probability to happening on some much later test. Low probability events happen all the time. Humans frequently misunderstand this.
Subjects: None
14 users liked this post.
2025-06-18T17:44:00
permalink Post: 11905421
I\x92m asking, \x93Whatever the flaw was that initiated these events, how did it remain un-known *until* it produced an accident?\x94 We test and re-test modern aircraft for every imaginable failure mode during the design, certification, and production process. We train and re-train techs, mechanics, flight crew, and everybody else that touches the airplane to be sure a high level of performance and safety is not compromised. Think bigger. Think about \x93the system\x94 as a whole. Apparently the system missed something. What are we not seeing?
Such a passage-of-time software issue wouldn't show up in most (or possibly any) testing scenarios. It is the sort of issue that robust QA and static code analysis are designed to catch. But in at least two separate systems on the 787 it has not been caught prior to software shipping. Meanwhile, every new technical post demonstrates the myriad ways in which non-software potential causes are mitigated by redundant design.
The odds of two (or more) redundant mechanical systems failing in precisely the same way at precisely the same moment are very, very small. The odds of identical software on two (or more) redundant systems reaching a passage-of-time bug at precisely the same moment are, by contrast, very much higher. True redundancy would require different software on each redundant sub-system.
Subjects: None
3 users liked this post.
2025-06-19T12:25:00
permalink Post: 11905983
I\x92d be very interested to know the current state of those switches, if they\x92re recoverable.
Subjects: None
5 users liked this post.
2025-06-19T16:12:00
permalink Post: 11906161
this isn\x92t the type of switch fitted to the 787 as a fuel control switch, totally irrelevant but has generated yet more nonsense. The switches are spring loaded (or so it feels) in addition to having a massive block to prevent inadvertent operation in either direction. Anyone suggesting they could be accidentally \x93knocked off\x94 is so clueless about their operation it\x92s actually painful to rebut
That's a TL series switch with 4 poles (the "4" in "4TL"), a "type D" lock (meaning locked out of centre position per the Honeywell data sheet - the "D" in "3D." This is a photo of one:
You can find the manufacturer's datasheet here: https://octopart.com/datasheet/4tl83...ywell-25749542
Problems with critical switches aren't new on 787-8s. For instance, in addition to the SAIB above, there's this AD: https://www.federalregister.gov/docu...pany-airplanes
Looking at the photo above, it isn't just wear that's potentially an issue, but foreign object impingement. There don't appear to be gaitors fitted to these switches in the 788, so the locking mechanisms are potentially susceptible to a build-up of material if not kept clean. There are a range of other failure modes possible, whilst the SAIB specifically describes found-in-the-field problems with these switches.
Yes, they're chunky, and very positive when new. That doesn't mean they shouldn't be discussed.
Subjects: Air Worthiness Directives Fuel (All) Fuel Cutoff
7 users liked this post.
2025-06-19T19:18:00
permalink Post: 11906289
In the history of jet transport aviation, both ETOPS and non-ETOPS operations, exactly how many simultaneous dual engine failures have there been, excluding pilot causal ones? I\x92d venture it\x92s zero. Even the old DC-9/Boeing 727 era had none. ETOPS is 40 years on and zero cases, to my knowledge. Modern twins are systematically divided into two separate and independent planes. My bet is all these neat theories based on arcane questions will boil down to some human causal event, excluding Boeing. They might contributory, as in the Delta 767 where the switch design contributed to pilot misaction, but design fault, vanishingly improbable.
ANA NH-985, a 787-8, suffered dual uncommanded engine shutdown just after air-ground transition. That was a TCMS "feature."
Baltic BT-139 likewise, resulted in an FAA AD to upgrade FADEC software on a whole bunch of P&W engines.
It isn't unheard of. It may not have been seen at rotation before.
Subjects: Air Worthiness Directives Engine Failure (All) Engine Shutdown FAA FADEC
1 user liked this post.
2025-06-19T21:09:00
permalink Post: 11906376
But the SAIB is clear - they can be assembled / installed incorrectly, resulting in no lockout protection, with actual failures / anomalies apparently found in the field.
None of which means they were the cause of this crash.
Subjects: None
2 users liked this post.
2025-06-20T00:36:00
permalink Post: 11906509
A good round-up of dominant themes, including this:
You may be at risk of assuming that the air/ground control logic is in some way hard-wired, as opposed to being a function of software. I don't believe we (yet) know this to be true.
We know there has been a bug in the Generator Control Unit software (an overflowing counter) that could lead to simultaneous shut down of all generators and a total loss of all AC power (the 248 days bug).
In the interests of completeness, we should perhaps also consider the possibility of some other previously-unknown software issue capable of creating an uncommanded dual engine shutdown. TCMS is the most likely candidate due to the deliberate separation of other systems from being able to achieve this outcome. The question then isn't whether there's some odd combination of input faults that would confuse TCMS into believing it were on the ground, but rather whether there's any way in which the software side could crash in such a way as to create an anomalous state within the system leading to engine failure. For instance, another overlooked software counter with an unwelcome failure mode.
Or even just a "dirty power supply" (cf all the reports of dodgy passenger-side electrics on this a/c) leading to spurious inputs and unexpected consequences.
Whatever is the cause will likely turn out to be have been a very low-probability event. But unless we have a TCMS expert who can state canonically that (say) the WoW sensor electrically disables TCMS when airborne (as opposed to merely being an input to the TCMS logic) then we cannot say with certainty that multiple inputs would have to have failed / been corrupted in order to reach the end state of this flight.
We know there has been a bug in the Generator Control Unit software (an overflowing counter) that could lead to simultaneous shut down of all generators and a total loss of all AC power (the 248 days bug).
In the interests of completeness, we should perhaps also consider the possibility of some other previously-unknown software issue capable of creating an uncommanded dual engine shutdown. TCMS is the most likely candidate due to the deliberate separation of other systems from being able to achieve this outcome. The question then isn't whether there's some odd combination of input faults that would confuse TCMS into believing it were on the ground, but rather whether there's any way in which the software side could crash in such a way as to create an anomalous state within the system leading to engine failure. For instance, another overlooked software counter with an unwelcome failure mode.
Or even just a "dirty power supply" (cf all the reports of dodgy passenger-side electrics on this a/c) leading to spurious inputs and unexpected consequences.
Whatever is the cause will likely turn out to be have been a very low-probability event. But unless we have a TCMS expert who can state canonically that (say) the WoW sensor electrically disables TCMS when airborne (as opposed to merely being an input to the TCMS logic) then we cannot say with certainty that multiple inputs would have to have failed / been corrupted in order to reach the end state of this flight.
Subjects: Dual Engine Failure Engine Failure (All) Engine Shutdown Generators/Alternators TCMA (Air-ground Logic) TCMA (All)
4 users liked this post.
2025-06-20T06:18:00
permalink Post: 11906620
TCMA can't be disabled electrically. It's just software, and all of the hardware involved serves other functions which are still needed while in the air. For example, the FADECs would command the HPSOV closed in case of N2 overspeed. That would have the exact same effect as TCMA.
Subjects: Air Worthiness Directives Dual Engine Failure Engine Failure (All) Engine Shutdown FAA FADEC High Pressure Shutoff Valve TCMA (All)
4 users liked this post.
2025-06-21T15:23:00
permalink Post: 11907840
That doesn\x92t make sense at all. 53% would be around 53t which is reasonable for that flight.
A full fuel load of 100t would have brought them by far over the MTOM as there is only around 8t of load left with full fuel of 100t. My guess with 242 POB and some baggage would be at least around 24t of load. Furthermore the fuel burn for tankering almost 50t for 9h would be enormous.
EDIT: Sailvi767 was quicker\x85
A full fuel load of 100t would have brought them by far over the MTOM as there is only around 8t of load left with full fuel of 100t. My guess with 242 POB and some baggage would be at least around 24t of load. Furthermore the fuel burn for tankering almost 50t for 9h would be enormous.
EDIT: Sailvi767 was quicker\x85
The plane carried almost 1,25,000 litres of fuel, and due to the high temperature, there was no chance of saving anyone.
I have no idea if he actually knew how much fuel was on board, or whether he'd simply been told it was "full" and somebody had looked up the capacity (126k litres) and he then quoted that "full" number. Or whether, in fact, there was an excess of fuel. But since that seems so far the only public statement on fuel load, it shouldn't just be dismissed out of hand.
2025-06-21T18:31:00
permalink Post: 11907969
This has been setting a bit wrong with me for the entirety of the thread. When people say "software" they have an image in their head, and in this case the image is rather unhelpful if not outright wrong.
When we say "Software" we mean things like this:
This is the type of software we are usually subject to in our everyday lives basically everywhere. Your phone, your fridge, your oven, your water heater, your car, etc. pp. ad nauseam.
In case of the Safran FADEC 3 this is not actually what we're dealing with. It uses something called an FPGA (Field Programmable Gate Array) which is a very different beast to what we are used to dealing with.
In a normal computer we simply translate code like above into something less human readable but instead interpretable by a microchip or the like. This "machine code" is then run sequentially and does whatever it is meant to do (hopefully). If we were to expand our little happy check above with another condition then our computer would sequentially check these conditions. This opens up a lot of hoopla about timing, especially when interacting with the real world.
Let's say you want to track N1 and N2 in an engine. You'd have to make a measurement, that value would have to go through an AD converter, which writes it to some (likely volatile) storage, where it is then accessed by some sort of (C)PU, transferred to yet another piece of volatile memory and then used in the computation. This takes time because you can't do those things within the same clock cycle.
Unsurprisingly this is rather inconvenient when dealing with the real world and especially when dealing with volatile physical processes that need monitoring. Like a modern turbine engine.
Enter the FPGA.
While it is programmable what that actually means is that (at a very high level) you can build a thing called a truth table, that means a definitive mathematical mapping of input states to output states. Unlike our sequential CPU driven system an FPGA will be able to perform its entire logic every time it is asked to do so. We don't have to wait for our happy check to perform any other check.
This is very useful for our Turbine Engine, because now we can verify that N2 is smaller than X without delaying our check that the Throttle Control Lever setting is within acceptable distance to N1 and N2, while also creating the appropriate output for all the different bypasses and bleed valves in a modern engine, and so on. The Safran FADEC 3 does this "up to 70 times per second" as per the vendor.
In order to be manageable the actual FADEC consists of multiple different pieces, many of which are FPGAs. At least two are used for so called "input conditioning". This is where we deal with converting messy real world values from sensors subject to real physics into nice and clean numbers. Here our values from the Throttle Control Levers come in, the signal from our N1 and N2 sensors, WOW signal, and on, and on, and on.
This is then changed into logic level signals provided to a different set of FPGAs. Lacking an actual schematic I suspect that this is what sometimes is referred to as "channels". The channel could consist of both a signal conditioning stage and then a logic stage or (more likely) redundant signal conditioning stages feed separately into separate FPGA "channels" are evaluated and then the end result is likely put into yet another FPGA for control output generation.
Why is this important?
Because it is basically impossible for a "bug" to exist in this system. These systems are the epitome of a dumb computer. They do _precisely_ what they are meant to do. The TCMA is simply a set of conditions evaluated just like any other condition in those FPGAs. If there is an "error" then it is in the requirements that lead to the truth table used by these devices but never in the truth table itself. In fact these types of computation device are used precisely because of that very property.
So when people say "the FADEC software" (ie TCMA) has "failed" or "has a bug" what they're really saying is:
Regards,
Justus
When we say "Software" we mean things like this:
This is the type of software we are usually subject to in our everyday lives basically everywhere. Your phone, your fridge, your oven, your water heater, your car, etc. pp. ad nauseam.
In case of the Safran FADEC 3 this is not actually what we're dealing with. It uses something called an FPGA (Field Programmable Gate Array) which is a very different beast to what we are used to dealing with.
In a normal computer we simply translate code like above into something less human readable but instead interpretable by a microchip or the like. This "machine code" is then run sequentially and does whatever it is meant to do (hopefully). If we were to expand our little happy check above with another condition then our computer would sequentially check these conditions. This opens up a lot of hoopla about timing, especially when interacting with the real world.
Let's say you want to track N1 and N2 in an engine. You'd have to make a measurement, that value would have to go through an AD converter, which writes it to some (likely volatile) storage, where it is then accessed by some sort of (C)PU, transferred to yet another piece of volatile memory and then used in the computation. This takes time because you can't do those things within the same clock cycle.
Unsurprisingly this is rather inconvenient when dealing with the real world and especially when dealing with volatile physical processes that need monitoring. Like a modern turbine engine.
Enter the FPGA.
While it is programmable what that actually means is that (at a very high level) you can build a thing called a truth table, that means a definitive mathematical mapping of input states to output states. Unlike our sequential CPU driven system an FPGA will be able to perform its entire logic every time it is asked to do so. We don't have to wait for our happy check to perform any other check.
This is very useful for our Turbine Engine, because now we can verify that N2 is smaller than X without delaying our check that the Throttle Control Lever setting is within acceptable distance to N1 and N2, while also creating the appropriate output for all the different bypasses and bleed valves in a modern engine, and so on. The Safran FADEC 3 does this "up to 70 times per second" as per the vendor.
In order to be manageable the actual FADEC consists of multiple different pieces, many of which are FPGAs. At least two are used for so called "input conditioning". This is where we deal with converting messy real world values from sensors subject to real physics into nice and clean numbers. Here our values from the Throttle Control Levers come in, the signal from our N1 and N2 sensors, WOW signal, and on, and on, and on.
This is then changed into logic level signals provided to a different set of FPGAs. Lacking an actual schematic I suspect that this is what sometimes is referred to as "channels". The channel could consist of both a signal conditioning stage and then a logic stage or (more likely) redundant signal conditioning stages feed separately into separate FPGA "channels" are evaluated and then the end result is likely put into yet another FPGA for control output generation.
Why is this important?
Because it is basically impossible for a "bug" to exist in this system. These systems are the epitome of a dumb computer. They do _precisely_ what they are meant to do. The TCMA is simply a set of conditions evaluated just like any other condition in those FPGAs. If there is an "error" then it is in the requirements that lead to the truth table used by these devices but never in the truth table itself. In fact these types of computation device are used precisely because of that very property.
So when people say "the FADEC software" (ie TCMA) has "failed" or "has a bug" what they're really saying is:
The conditions that were experienced in the real world by the system were incorrectly assessed in the system requirements and lead to an undesired output state when taking into account the real world result we would have preferred.
A bit of a mouth full, granted, but an important mouth full. This simply wouldn't be someone missing a semicolon somewhere in a thousand lines of code.
Regards,
Justus
Subjects: Air Worthiness Directives Engine Failure (All) Engine Shutdown FAA FADEC TCMA (All)
2025-06-21T18:33:00
permalink Post: 11907973
What I think we can say is, if the a/c were fuelled as the minister suggested then there\x92d be a major weight issue for the a/c that day. Absent any other info we can say little else.
Subjects: None
2025-06-30T03:42:00
permalink Post: 11913342
This has also been touched upon earlier in the thread, but it rather seems the cut-off switches are in the same LRU, in close proximity, using the same connector and goes through the same wiring harness. No one was able to say whether it works purely by digital signaling, and goes through any common software, or if it is duplicated by purely direct signaling. There might be numerous failure modes of the cut-off switch design, it is obviously very, very robust and overall sound, since dual failures here have never happened, but this is alredy an outlier event.
That's a pretty big "if" but here's the patent drawing:
Subjects: Fuel (All) Fuel Cut Off Switches Fuel Cutoff TCMA (All)
2025-06-30T12:29:00
permalink Post: 11913592
We know that the right-hand GEnx-1B was removed for overhaul and re-installed in March 2025 so it was at \x93zero time\x94 and zero cycles, meaning a performance asymmetry that the FADEC would have to manage every time maximum thrust is selected. If the old engine was still on the pre-2021 EEC build while the fresh engine carried the post-Service Bulletin software/hardware, a dual \x93commanded rollback\x94 is plausible. A latent fault on one channel with the mid-life core can prompt the other engine to match thrust to maintain symmetry, leading to dual rollback.
Subjects: Dual Engine Failure Engine Failure (All) FADEC
1 user liked this post.
2025-06-30T14:04:00
permalink Post: 11913652
Posters may like to read this old (2016) pprune thread on 787 engine failure procedures:
787 engine failure procedure
Some interesting comments about how a combination of ATM and derate can lead to some pretty surprisingly poor outcomes, coupled with Boeing advice not to advance thrust levers or engage TOGA.
(edited for poor spelling)
Some interesting comments about how a combination of ATM and derate can lead to some pretty surprisingly poor outcomes, coupled with Boeing advice not to advance thrust levers or engage TOGA.
(edited for poor spelling)
Last edited by skwdenyer; 30th Jun 2025 at 14:40 .
Subjects: Engine Failure (All) TOGA