I spent several years writing software for functional safety systems in refineries. The kind of systems governed by standards like PN-EN 746-2, where a failure doesn’t just cost money. It costs equipment. Sometimes worse. Testing was a substantial part of the problem, and the problem was this: you can validate every requirement in the standard, run every test case your team can think of, get a clean pass report, and still have failure modes you never anticipated. Not because you were careless. Because the gap between what you tested and what actually happens in the field is structural. It is built into the way we simulate, model, and verify embedded systems.
That gap has a name in the research literature. Reway et al. (2020) published a test method specifically for measuring the simulation-to-reality gap of camera-based object detection algorithms for autonomous driving at the IEEE Intelligent Vehicles Symposium. Mirletz, Park, and Quinn (2015) documented what they called the “reality gap,” the well-known problem that “those who try to apply controllers learned in a simulator to hardware” face when the simulation results do not carry over. Steinhauser et al. (2024) categorized and contextualized “these discrepancies within the structure of hardware-in-the-loop” for automated driving functions. Brogle et al. (2019) demonstrated that including hardware in the loop narrows this gap, but they were careful to note that it narrows it, not eliminates it.
This article is about that gap. Not about HIL being useless. HIL is essential. It catches bugs that would otherwise reach production. It saves lives. But it also creates a specific kind of blind spot, one that is particularly dangerous because it comes wrapped in a test report that says PASS. The gap is not a secret. Engineers on forums describe it daily. Academics publish papers about it. HIL vendors themselves warn about it in their own marketing material. And yet, across industries, test teams continue to treat a clean HIL run as evidence that a system is ready for the field. It is not. Here is why.
Let me start with what the forums actually say, because this is where the gap lives in practice, not in theory.
”It Works in MAX. It Fails in My Code”
Spend a few hours on the NI Community forums, where engineers who build and maintain automated test systems discuss their daily problems, and a pattern emerges that is impossible to unsee once you recognize it. The single most common complaint is not about missing features or licensing costs or hardware failures. It is about a specific kind of broken assumption: the interactive test panel says the instrument is fine, but the moment you run the same commands programmatically, everything fails.
One engineer described debugging a Chroma 16502 LCR meter over serial: “I am able to get response to serial command from Chroma 16502 Ohmmeter, but when I create a VISA write and read in the LabVIEW then it errors out as Read Timeout” (NI Forum, Thread 4339496). What was happening? NI MAX, the interactive configuration utility, automatically appended termination characters to serial commands. LabVIEW code did not. The same instrument, the same commands, the same computer. The interactive validation environment added a hidden assist that the automated test environment did not provide. The test panel showed the device communicating correctly. The automated script showed timeout. Neither was wrong. They were simply testing two different environments.
This is not an isolated bug. It is a category. An analysis of 117 NI Forum posts – manually coded for test-environment mismatch patterns – found that the majority of recurring complaints center on this specific asymmetry: the interactive validation tool reports success, the programmatic execution fails. The phrase “It works in MAX. It fails in my code. I haven’t changed anything” has become a shorthand among NI ecosystem engineers for the verification environment gap. And it highlights something fundamental about simulation and testing: the test environment itself is part of the system. Change the environment, even in a way that seems trivial, and you are testing a different thing.
Another engineer, working with an NI-8451 USB-to-SPI adapter, captured the essence of the problem in a single thread: “when using 8451 Test Panel I get random results, only once i got expected results and i captured this moment (i cant reproduce it somehow)” (NI Forum, Thread 2802750). The interactive test panel is supposed to be the debugging tool, the ground truth. When even the ground truth produces random, non-reproducible results, what exactly is the test validating? The engineer captured the correct output once. Once. And could never reproduce it. The test tool itself was non-deterministic.
Reading through these threads, I kept thinking about the refinery. There was a safety shutdown system I had worked on. It validated against all the requirements in the standard. Every test case passed. The logic was sound. What we had not modeled, what no one had thought to model, was that a particular combination of sensor inputs, arriving in a specific sequence with a specific inter-arrival timing that only occurred under one particular operating condition, would cause a state transition that the system was not designed to handle. The condition simply had never occurred during testing because the test scenarios were derived from the standard, and the standard did not describe that scenario. The standard described what should happen. It did not describe what could happen.
The forums are filled with variations on this theme. The analysis of 117 NI Forum posts also surfaced the TestStand callback pattern: callbacks fail silently, tests continue, bad units ship. There is the deployed cRIO application that “works fine” when tested from the development host over Ethernet, but fails within hours when running autonomously at startup: “When testing the system with a computer (via ethernet cable) the acquisition works fine. But when I run the application at startup of the cRIO, I noticed after a while that the system is not working anymore” (NI Forum, Thread 3850999). There is the executable that ran for seven years in development mode, and then silently dropped critical library files the first time it was compiled for deployment: “The source code runs with no discrepancies, and has been used running source for the past seven years. I created and ran the executable, and here is the order of how the errors stack up… a whole bunch of vi.lib files are not making it into the executable” (NI Forum, Thread 3905792). Seven years of successful operation in development mode. First compiled executable: broken.
These patterns recur across industries, toolchains, and years. The gap is persistent, not exceptional. What makes it dangerous is that the test report does not know about it. The report says PASS. The report does not say “PASS, assuming the termination characters are appended, assuming the execution timing matches the interactive debugger, assuming the deployed target behaves identically to the development host, assuming no library files were silently dropped during compilation.” The report cannot tell you what it did not test.
The Verification Environment Gap: A Taxonomy
When these field failures are aggregated, what emerges is not a single “HIL gap” but a multi-dimensional taxonomy of verification mismatches:
| Forum Symptom | Actual Gap Category |
|---|---|
| MAX validation succeeds, API script fails | Tool-to-Code Gap |
| UI accepts value, hardware silently ignores it | Closed-Loop Verification Gap |
| ”Lightbulb mode” (debugger) masks error | Debug-to-Runtime Gap |
| Source code runs, compiled executable drops libraries | Build / Deployment Gap |
| Enterprise antivirus blocks vendor DLLs | IT / Endpoint Security Gap |
| VFD generates EMI causing RS485 timeouts | EMC / Site-Installation Gap |
| Module leaks 0.5 GB over seven days | Endurance / Soak-Test Gap |
Each of these is a distinct failure mode, but they share a common root: the PASS verdict was produced in a verification environment that did not match the operating environment in one critical dimension. The rest of this article examines several of these categories in detail.
Ghost Settings: When the Test Tool Shows What You Asked For, Not What You Got
There is a particular category of failure that reveals something about the relationship between test tools and reality. Neil Pate, posting on the NI Forum about a data acquisition module, described a discovery he made entirely by accident: “What I did not realise is that this module does not have a user definable excitation current, it is fixed at 100 uA. Oddly though MAX lets you change the excitation current, and it was only when I actually measured the current did I realise that the current does not change. (Why MAX lets you change it, I do not know, seems like a bug to me…)” (NI Forum, Thread 1000494).
The primary configuration tool for NI hardware presents a setting that the hardware physically cannot change. You can type in a new value. The tool accepts it. It displays it. It reports no error. The hardware simply ignores it, delivering its fixed 100 microamps regardless. Neil only discovered this because he measured the actual current with a multimeter. How many test systems are running right now with configuration parameters that exist only in software? How many test reports carry the signature of settings that were never applied to the hardware?
This is not a one-off. It is a systematic failure mode of test infrastructure that lacks closed-loop verification: the test tool shows you what you asked for, not what you got. Neil’s multimeter was the difference between a passing test and a correct test. Most test systems do not include a multimeter in the feedback loop.
What makes ghost settings dangerous, as opposed to merely annoying, is that they erode the fundamental premise of automated testing. The premise is that the test infrastructure faithfully represents the system under test. If the test tool can display a configuration that the hardware silently ignores, the tool is not faithfully representing anything. It is presenting a fiction. The test report that says PASS is reporting on the fiction. The real hardware, operating under the real configuration, was never tested. The team believes they have validated a system running at a specific excitation current. They have validated nothing of the kind. They have validated a user interface element that accepted their input.
This is not unique to NI MAX. It is a category error that affects any test infrastructure where the configuration layer and the hardware layer are not continuously cross-validated. If your test framework allows you to set a parameter and reports success without verifying that the parameter was actually applied to the hardware, you have the same problem Neil had. You just may not have measured it yet.
The timing gap is another persistent category. An engineer described a classic debugging paradox: “The code works in lightbulb mode because you are actually supposed to wait for a response before issuing the next command. Since lightbulb mode slows the code down immensely, it gives the machine time to get ready for the next command” (NI Forum, Thread 4034358). Lightbulb mode is LabVIEW’s step-by-step debugger. It executes each node sequentially, with a visible pause between operations. Normal execution runs at full speed. The instrument, in this case, had undocumented timing requirements. It needed a pause between commands that the normal execution speed did not provide. The debugger, by slowing everything down, satisfied the requirement by accident.
This means that the debugging tool, the very mechanism you use to verify correctness, masks the bug. Every time you step through the code looking for problems, the code works. Every time you run it at full speed, it fails. The test tool and the real execution environment are not just different. They are actively contradictory. One hides what the other reveals.
The same pattern appears in race conditions. An engineer on the NI Forum described a problem that only appeared in compiled executables, never in the development environment: “It sound like the issue is that you have a race condition. Essentially what is happening is that two things are happening in parallel that should be happening sequentially. Inserting the breakpoint changes the order in which LV executes the code and prevents the problem from occurring” (NI Forum, Thread 2866912). The breakpoint changes the execution order. The execution order masks the race condition. The debugging environment produces correct behavior by construction. The production environment does not.
Even more striking is the case where the vendor’s own chief engineer cannot reproduce a field failure. Marcos Kirsch, Chief Software Engineer for NI Driver Software, responded to a report affecting three separate PXI systems with this: “No root cause has been determined as we haven’t been able to reproduce the issue internally” (NI Forum, Thread 4431744). Three customers, three systems, the same failure, the same hardware configuration. The vendor confirmed the issue existed in the field. But NI’s internal lab, with what one assumes is the best available test equipment and the engineers who built the product, could not trigger it. The failure mode was real. It simply did not exist in NI’s lab environment.
The root cause, when eventually identified, was not the driver. It was a third-party security agent: “Our IT vendor tested temporarily disabling SentinelOne Agent on the PC that was giving us this issue, and the connection problem went away immediately” (same thread). SentinelOne, an endpoint security product, was interfering with NI’s DLL loading mechanism. This is not a condition any hardware vendor’s test lab would simulate. It is a condition that exists in real enterprise IT environments, on real customer machines, producing real failures that the vendor’s lab, no matter how well equipped, would never catch.
What makes this case particularly instructive is the chain of assumptions it exposes. The vendor assumed the driver would load correctly on a standard Windows installation. The customer assumed the vendor had tested for enterprise security software. Neither was correct. Both were reasonable. The gap between them is exactly the gap that a HIL test, no matter how comprehensive, cannot close, because the gap exists in the environment, not in the device under test.
This is the essential asymmetry of the simulation-to-reality gap. The device under test is known. It has specifications, data sheets, test procedures. The environment is unknown. It has antivirus software, grounding schemes, firmware versions, driver conflicts, neighboring equipment generating EMI, ambient temperatures, vibration profiles, operator behavior patterns. You cannot test what you do not know exists. But it exists, and it will affect your system. The gap is not a bug in the simulation. It is a property of the relationship between any simulation and any reality, and it can only be managed, never eliminated.
This is also why the “just test more” advice that pervades engineering culture is insufficient. Testing more of the same things, under the same conditions, finds the same bugs. It does not find bugs that require conditions absent from the test environment. To find those bugs, you must change the test environment, and to know what to change, you must first understand what your current environment is missing. This is a different kind of work than writing test cases. It is investigative work. It requires looking at the system from the outside, asking what assumptions are baked into the test infrastructure, and deliberately breaking those assumptions one at a time.
Real-world electrical conditions are another domain where the simulation gap opens wide. A MODBUS RTU energy analyzer communicating over RS485 in an industrial setting worked perfectly in the lab. Deployed next to a variable frequency drive controlling a motor, the communication began failing. The engineer who debugged it noted: “Previously I made the program in LabVIEW… that work fine but the problem was motor controlled from 80-4000RPM” (NI Forum, Thread 4326047). The VFD was generating electromagnetic interference on the RS485 line. The lab had no VFD. The lab test passed. The field installation did not.
Another engineer, diagnosing sensor noise, identified 50 Hz mains interference corrupting readings: “The 50 Hz… I would suspect that the frequency of your line voltage is what’s being measured due to some sort of grounding error or line noise interference.” Fifty hertz is the power grid frequency in Europe. It is everywhere. It is also the kind of environmental condition that a well-grounded lab bench filters out and a production floor injects back in through grounding paths, cable routing, and proximity to motors.
An engineer working on a high-temperature FPGA programming application described the challenge directly: “I am trying to create a system to test this device for high temperature applications. The end application is for programming a Xilinx FPGA.” High temperature. Not room temperature. Not the temperature of the test lab. The temperature the device will actually experience. Labs are air-conditioned. Production environments, by definition, are not, or at least not to the same specification. Testing at 22 degrees Celsius tells you very little about behavior at 70 degrees. Every electronics engineer knows this. The testing infrastructure rarely accounts for it.
Memory leaks represent a particularly insidious version of the simulation gap because they are invisible to short-duration tests. A DCAF Modbus module was found to leak approximately 0.5 GB per day in a production deployment running on a 200 ms loop: “The leak took about 7 days, which we calculated to be around 0.5GB/day. Running on a loop time of 200ms” (NI Forum, Thread 3830099). NI confirmed the leak on GitHub. But consider what it would take to detect this in a standard test environment. A test lasting one hour would leak about 20 MB, which on a system with gigabytes of RAM is imperceptible. A test lasting one day is rarely performed in a development sprint. A test lasting seven days is essentially never performed. The production deployment runs for months. The lab test runs for minutes or hours. The memory leak that eventually crashes the system is invisible to every test the team runs, and the team concludes the system is stable.
Even a minimal project exhibits the same behavior: “I generated a new Execution Service project with just a couple of tags in it. Running this continuously on my cRIO, I get a leak of approx 50 MB over a 16 hour period” (same thread). A bare-minimum configuration leaks 50 MB every 16 hours. There is no test short enough to miss this that is also long enough to catch it. The test duration required to observe the failure exceeds the test duration any reasonable development process allocates.
The Academic Proof: Measuring the Gap
This brings us to the academic literature, which has quantified exactly this gap in domains where the consequences are measured in millions of dollars and human lives. The most striking finding comes from the Air Learning project, a research platform for training deep reinforcement learning policies for autonomous UAV navigation. Krishnan, Boroujerdian, Fu, Faust, and Reddi (2021), in a paper published in Springer’s Machine Learning Journal, trained a navigation policy in simulation and then tested it in hardware-in-the-loop configuration, with the policy running on an embedded Raspberry Pi while the flight environment remained simulated. The results were stark. The HIL setup exposed what desktop-only testing missed.
Thirty-seven point seven three percent. Not 5%. Not 10%.
“We find that the trajectories on an embedded Ras-Pi are vastly different from those predicted on a high-end desktop system, resulting in up to 40% longer trajectories in one of the environments.” Forty percent. A policy that navigated perfectly in desktop-only simulation produced simulated trajectories up to 40% longer when executed through the embedded target compute platform in HIL, because the desktop simulation did not account for the computational latency of the embedded processor.
The researchers did something instructive here. Instead of simply reporting the gap, they closed it: “Training the policy with artificial delays allows us to minimize the hardware gap (discrepancy in the flight time metric reduced from 37.73% to 0.5% ).” They injected simulated latency into the training environment that matched the real latency of the embedded processor. The simulation-to-reality gap fell from nearly 38% to essentially zero. They did not improve the algorithm. They did not upgrade the hardware. They made the simulation less perfect, more realistic, and the bridge between simulation and reality closed almost completely.
The lesson of Air Learning is not that simulation is broken. It is that simulation is only as good as the gap between what you simulate and what actually happens. The gap in this case was computational latency. The researchers identified it, measured it at 37.73%, injected it into the training environment, and eliminated 99% of the discrepancy. But here is the uncomfortable question: how many HIL test systems in production environments have had their simulation-to-reality gap measured? How many teams know what their gap is?
The answer, based on the forum data, is that systematic gap measurement is virtually absent. The NI Forum threads contain zero mentions of anyone quantifying, measuring, or even systematically characterizing the difference between their test environment and their deployment environment. Engineers describe the symptoms of the gap constantly. They debug the consequences of it. They work around it with polling loops and wait statements and configuration hacks. But the gap itself is invisible to their process. It is not tracked. It is not measured. It is not part of the test report. It is the thing that happens between the test report and the field failure, and it is nobody’s job to understand it.
Even Vendors Warn Against Treating Signal-Level Passes as System Validation
The HIL industry itself is increasingly aware of this problem. OPAL-RT, one of the major HIL platform vendors, published a series of unusually candid blog posts in mid-2026 about the limits of signal-level HIL validation. In a post on enhancing automotive powertrain confidence using HIL and SIL, the company observed: “A signal-level pass alone will miss what the vehicle actually does. Good criteria link low-level data to system outcomes. That reduces false confidence, which is a common failure in powertrain validation using simulation” (OPAL-RT, June 2026). A companion post, “6 Types of HIL Testing,” made the boundary point even more directly: “The right HIL test bench starts with the failure you need to expose.”
False confidence. A vendor describing its own product category as a source of false confidence when used incorrectly. This is not a criticism of HIL. It is a precise diagnosis of the failure mode: teams run HIL tests, see signal-level passes, and conclude the system is ready. The signals are correct. The system behavior is not. The difference between a signal and an outcome is exactly the difference between a test that validates electrical characteristics and a test that validates whether the thing actually works.
The same post also noted: “Nominal operation proves only that the system works in its easiest state. Confidence comes from fault injection, degraded modes, and recovery checks. Many failures appear during edge conditions, charge events, and protective transitions” (OPAL-RT, June 2026). The easiest state. That is what most test suites validate. The system in steady state, under nominal conditions, responding to expected inputs. The real world does not operate in the easiest state. It operates at the edges: during startup, during shutdown, during fault recovery, during overload, during undervoltage, during firmware update, during sensor degradation, during every transition that the nominal test suite does not cover.
The vendor is warning against a common misuse: treating signal-level correctness as system-level validation. But the implications extend beyond any single vendor’s platform. If nominal-cycle testing produces false confidence, and if most teams test nominally because that is what their test plans specify, then false confidence is not an occasional mistake. It is the default outcome of the standard approach to HIL testing. The OPAL-RT post is essentially warning its own customers: you are using our product to validate the easiest state, and that is not the same as validating the system. The fact that this warning appears on a vendor’s marketing blog rather than on a test report template is itself revealing. Nobody is required to read it. Nobody is required to act on it. It is advice, and advice is optional.
This maps directly to what we observed in the refinery: the safety system that passed every requirement in the standard had never been tested in the specific operating condition that triggered the failure. The standard described nominal behavior and known fault conditions. The failure mode was neither. It was an emergent property of the interaction between sensor input timing and the system’s internal state machine, a combination that no one had anticipated and no standard required to be tested.
The economic cost of false confidence is remarkably difficult to calculate, which is precisely why it persists. A test that passes costs nothing visible. A field failure costs everything: warranty claims, recall logistics, regulatory fines, lost contracts, engineering time diverted to root-cause analysis, reputational damage that takes years to repair. But the causal chain between the passing test and the field failure is rarely documented. The test report that said PASS is filed away. The failure investigation focuses on the immediate cause: the overflow, the race condition, the memory leak. Nobody traces the failure back to the test that should have caught it but did not, because the test environment did not include the condition that triggered it.
Why Nobody Owns the Gap
This is the deepest problem exposed by the forum data. It is not that engineers do not know the gap exists. They do. Every engineer who has debugged a field failure that never appeared in the lab knows the gap exists. The problem is that the gap is nobody’s job. The test team’s job is to write and execute test cases, typically derived from requirements. The development team’s job is to fix bugs and ship features. The quality team’s job is to track defects and manage releases. Nobody’s job description includes the phrase “quantify and close the verification-environment gap.” And so it remains unquantified, unclosed, waiting silently in the space between the PASS report and the field deployment.
So what is the practical response to all of this?
How to Close It
HIL is one instance of a broader verification-environment gap. The same structural failure appears whenever the environment that produces the PASS report differs from the environment in which the system must operate. What follows applies whether you are testing a refinery safety system, a UAV navigation policy, or an automotive powertrain. Model, then fuzz, then verify on real hardware. In that order. Each step addresses a different part of the gap, and skipping any step leaves a specific kind of blind spot.
1. Model first. Build a simulation that captures everything you know about the system: the nominal behavior, the known fault conditions, the expected input ranges, the timing constraints, the communication protocols. The model will be incomplete. That is acceptable. The purpose of the model is not to replace reality. It is to provide a fast, repeatable, deterministic environment where you can test everything you can think of. Run every requirement from the standard against the model. Run every combination of inputs. Run the edge cases you have documented. The model will catch a large class of errors quickly and cheaply. It will also reveal gaps in your understanding, because building a model forces you to be explicit about assumptions that are implicit in the requirements document. Every time you cannot model something precisely, you have identified a gap. Write it down. That gap is now part of your test documentation. It is evidence of what you know you do not know.
2. Then fuzz. Take the model and feed it randomized inputs at randomized timing. Run it for hours. Run it for days if you can. Fuzzing finds the bugs you did not think to look for because it makes no assumptions about what inputs are valid or what timing is expected. A concurrency-aware test harness with randomized timing and operator-input sequences might have exposed callback races and silent failures of the kind described in the forum evidence earlier, provided the test oracle monitored illegal state combinations rather than only final pass/fail status. A keystroke-level fuzzer feeding rapid edit sequences could have probed for timing vulnerabilities of the kind that produced the failures discussed throughout this article. Fuzzing is not subtle. It is brute force. It works precisely because it does not share your assumptions about what the system should do.
The NI Forum is a publicly readable trace of a problem that, in other toolchains, happens behind closed doors. The memory leak that took seven days to detect would have been found by a fuzzer running on a weekend. The intermittent SPI read failures on the NI-8451 would have been exposed by a fuzzer that logs every anomalous result instead of averaging them away. The timing-dependent serial communication failures that only appeared at full execution speed would have been visible in a fuzzing run that varies inter-command delays. The EMI interference from the VFD would not have been found by fuzzing a software model, which is exactly why the third step exists.
3. Verify on real hardware. Not just once. Not just in the lab. In the environment the system will actually operate in, or as close to it as you can get. This is where you discover that the excitation current setting in MAX does not actually change the hardware. It is where you discover that the enterprise antivirus blocks the DLL. It is where you discover that the grounding scheme on the production floor injects 50 Hz noise into your sensor readings. It is where you discover that the deployed cRIO application that ran fine in the lab crashes after a few hours of autonomous operation. The model and the fuzzer run in a controlled environment. They cannot discover conditions that do not exist in the lab. Only the real hardware, in the real environment, can close that loop.
The Air Learning researchers understood this implicitly. They did not stop at modeling. They did not rely on simulation to prove their policy was correct. They deployed to real hardware, measured the gap at 37.73%, injected the missing condition back into the simulation, and iterated until the gap closed to 0.5%. That is the loop. Model, fuzz, real, repeat. Each iteration narrows the gap. Each iteration produces evidence of what was missed and what was corrected. Each iteration makes the next test more trustworthy.
A free workflow diagnostic kit is one way to start mapping what your current test environment actually covers versus what it assumes. But regardless of the tool, the discipline is the same: measure the gap, close it iteratively, and produce evidence at each step that you understand what remains unmeasured.
There is a final piece that makes this cycle more than just good engineering practice. It is evidence. Not the test report from the model. Not the log from the fuzzer. Not the pass/fail verdict from the HIL rig. The evidence chain that connects all three. When an auditor asks whether your system was adequately tested, a HIL report that says PASS is not enough. What the auditor needs to see is that you understood the gap: that you identified what the model missed, that you characterized what the fuzzer found, that you verified on real hardware what conditions the simulation could not capture, and that you can trace a requirement to a test, a test to a result, and a result to a real hardware execution under conditions that reflect the operational environment.
This is not a testing problem. It is an evidence problem. Building a trusted evidence chain across model, fuzz, and real hardware is what transforms a test report from a PASS verdict into a defensible record of what was tested, what was missed, and why. The test passes. The question is what the test did not simulate. And whether you can prove you know the difference.
The gap has been quantified in specific domains. 37.73% in UAV navigation policy transfer from desktop to embedded hardware. Forty percent longer trajectories. Memory leaks invisible below seven-day test durations. Configuration settings that the hardware ignores. Callback failures that the test executive silences. Serial commands that work interactively and fail programmatically. Debugging tools that mask race conditions by slowing execution. Enterprise security software that blocks vendor drivers. VFDs that inject EMI into communication lines. Each of these is a manifestation of the same structural problem: the test environment is not the operating environment, and the difference between them is where failures hide.
The practical question for any team running HIL tests today is not whether their simulation is accurate. It is whether they have measured how inaccurate it is. If you cannot identify or bound the main dimensions of your simulation-to-reality gap, your PASS report is incomplete evidence – not proof of field readiness. And if you treat incomplete evidence as proof, the difference between a PASS and a field failure is only a matter of time.
Your HIL test passed. Your system has not yet flown. The question is not whether your test was thorough. The question is what your test did not simulate, whether you know what it was, and whether you will discover it before your customer does.
Sources
Academic papers
Krishnan, S., Boroujerdian, B., Fu, W., Faust, A., & Reddi, V.J. (2021). “Air Learning: A Deep Reinforcement Learning Gym for Autonomous Aerial Robot Visual Navigation.” Machine Learning (Springer).
Reway, F., Hoffmann, A., Wachtel, D., Huber, W., Knoll, A., & Ribeiro, E. (2020). “Test Method for Measuring the Simulation-to-Reality Gap of Camera-Based Object Detection Algorithms for Autonomous Driving.” IEEE Intelligent Vehicles Symposium (IV).
Mirletz, B.T., Park, I.W., & Quinn, R.D. (2015). “Towards Bridging the Reality Gap Between Tensegrity Simulation and Robotic Hardware.” 2015 IEEE/RSJ IROS.
Steinhauser, C., Ransiek, J., Langner, J., et al. (2024). “Contextualizing the Sim-to-Real Gap for Hardware-in-the-Loop Testing of Highly Automated Driving Functions.” 8th International Conference.
Brogle, C., Zhang, C., Lim, K.L., et al. (2019). “Hardware-in-the-Loop Autonomous Driving Simulation Without Real-Time Constraints.” IEEE Transactions on Intelligent Vehicles.
Industry sources
OPAL-RT Technologies. (June 2026). “6 Types of HIL Testing.” Blog post.
OPAL-RT Technologies. (June 2026). “Enhance Automotive Powertrain Confidence Using HIL and SIL.” Blog post.
Forum threads (NI Community)
All forum quotes are drawn from publicly accessible NI Community forum threads. Each quote is cited inline with the thread number where available. Key threads referenced: 4339496 (Chroma VISA timeout), 2802750 (NI-8451 SPI random results), 1000494 (MAX ghost excitation current), 4034358 (lightbulb mode timing mask), 2866912 (breakpoint race condition), 4431744 (SentinelOne DLL interference), 4326047 (MODBUS VFD EMI), 3830099 (memory leak), 3850999 (cRIO dev vs autonomous), 3905792 (seven-year dev mode executable).
Standards
PN-EN 746-2: Industrial Thermoprocessing Equipment – Safety Requirements for Combustion and Fuel Handling Systems (historically; superseded in the current European framework by EN ISO 13577-2 and EN ISO 13577-4).
Understand why HIL tests pass but real systems fail, and what can be done about the simulation-to-reality gap.