Realistic RF and System Testing
"Poor Testing leads to a Poor Product"
Connection drops, weak RF performance, high failure rates – why do many wireless and IoT devices work in the lab but fail in the field? Enrico Brinciotti and Martin Varga (Anritsu) explain common RF/EMC test mistakes and how realistic testing boosts product reliability.
Elektronik: You represent the viewpoint “Poor testing leads to poor products.” What exactly do you mean by that?
Enrico Brinciotti, EMEA Director of Engineering, Marketing & Business Development: When we say poor testing leads to poor products, we mean that insufficient validation inevitably translates into poor user experience and reduced product quality in the field. Proving that something works is relatively easy. The hard part is understanding how well it works under real conditions.
There’s often a mismatch between how products work, or appear to work, during development in controlled lab environments, and how well they work in the real world. In many cases the product doesn’t completely fail, but it may well perform below expectations, or underperform against competitors, which can be just as damaging commercially.
If testing is incomplete or superficial, issues remain hidden until the product is deployed and this means that the real stress-testing doesn’t happen until the product is out in the field. That’s when problems such as dropped connections, inconsistent performance, or user complaints start to appear.
Why do so many products fail in the field, even though they perform well in the lab?
Martin Varga, Wireless Team Leader: Many teams validate functionality only at a very simplistic level. This might involve for example connecting a device to a nearby access point, confirming that it works, and ticking the relevant box. But out in the real-world, applications are far more complex, and each one is unique. Signal strength will vary, or interference will disrupt operation. A product might work perfectly well on a desk next to a PC, for example, but place the same product a few meters away and it may fail in completely unexpected ways.
The underlying issue here is that too often teams only test whether a device works, not how well it works. If one fails to take both into account, you’re essentially blind to how the product will behave in the field. It also potentially means missed chances to optimize and improve the resilience of the product through testing and analyzing its true limits in real operation. The upshot is that this creates situations whereby problems only appear once customers start using the product in unpredictable real-world scenarios.
In your opinion, why do companies often underestimate the costs that insufficient testing will cause later?
Martin Varga: The costs of insufficient testing are often indirect and, may only become apparent later in the process, or as knock-on effects. By this time, it’s already too late to undo what has already been done. The most obvious example is failing certification, which means paying for the process again and delaying the product launch. But one oversight can cascade into multiple cost streams. Redesigns may be needed, along with additional engineering time, ultimately leading to missed market opportunities. All of these have their own costs, and all are avoidable.
Enrico Brinciotti: There is also a strong competitive dimension: even if a product works, poorer performance compared to competitors can lead to lost sales and reduced market share. Perhaps most importantly, there is the opportunity cost, where delays can mean losing market positioning altogether.
These costs are often unquantifiable, so companies instead tend to focus on the visible cost of test equipment yet underestimate the much larger downstream risks. Often, many of the biggest costs are “hidden losses”. This can manifest as delays, lost deals, or missed market windows. While they are hard to measure, they can be highly impactful and can cause lasting damage.
Where do you see the biggest quality problems when tests happen too late or are too superficial?
Enrico Brinciotti: The biggest issues typically arise in areas like RF performance and system integration. For example, interference between components such as Bluetooth and Wi-Fi, or poor antenna performance, can have a detrimental effect on the product’s overall performance. One of the key RF performance tests is receiver sensitivity testing. Commonly performed on cellular, Wi-Fi and Bluetooth devices, this test evaluates how increasing distance from a base station, Wi-Fi access point, or paired Bluetooth device affects communication quality and determines the point at which communication is no longer acceptable. Compared to a simple connection check, receiver sensitivity testing provides a clear and quantifiable performance measurement of communication reliability and range.
These issues are often not visible in basic functional testing but emerge when performance is pushed to real-world limits. If testing is done too late, they are discovered only after the design is largely finalized, making it much more expensive to resolve these issues after the fact.
Martin Varga: In many ways, it’s like building a skyscraper and then deciding to revisit the foundation at the end – it’s not impossible to fix it, but it is extremely costly and highly disruptive. In some cases, form or performance may have to be sacrificed to accommodate redesigns, which is far from ideal.
What misconceptions do you encounter most often when teams assess their wireless or EMC testing?
Enrico Brinciotti: One of the most common misconceptions is that using a certified module eliminates the need for further testing. Teams assume that because the module passed certification, the final product will also perform correctly. But a module is only one part of the whole. Integration introduces additional complexities and uncertainties, and extrapolating potentially misguided assumptions around certified modules can reduce visibility and control over system-level performance.
Martin Varga: Another misconception is equating “it works” with “it performs well”. There is a clear distinction between measuring performance and detail and merely verifying functionality.
Finally, there is also a broader awareness gap, particularly among newer players in the IoT space. Teams sometimes don’t fully realize that they may have a problem until it manifests in the field. No one is accusing teams of deliberately cutting corners, but these misconceptions mean that they simply don’t know what they should be testing or why it matters.
What really makes a good testing strategy for stable wireless performance and long battery life?
Martin Varga: Ultimately, the goal is twofold: reduce risk and optimize performance. A good testing strategy goes beyond basic validation and focuses on understanding performance in depth. Achieving this means testing across as many scenarios as feasible, including realistic conditions, extreme conditions, and hypothetical edge cases. Testing only for ideal scenarios in the lab builds an incomplete picture of true performance.
As well as understanding the limits of the product, this also allows you to understand why components and systems behave the way they do. Quantifying performance allows you to understand, for example, exactly when and why a connection degrades or fails. It also involves measuring how the device behaves across different environments, such as performance degradation with distance or interference. This is valuable work and can feed into future product development.
On the power side, it’s essential to measure consumption in different operating states, such as idle, active transmission, or poor signal conditions. Power consumption can vary significantly depending on network conditions, such as devices consuming much more energy when operating far from a base station or under weak signal conditions.
Without controlled testing, these variations are very difficult to detect and optimize. Using controlled environments like network simulators allows engineers to replicate these scenarios precisely and make design decisions based on a deep understanding of how they will play out.
How do automated and standardized tests help uncover errors earlier and more reliably?
Enrico Brinciotti: Automation and standardization bring consistency and repeatability to the testing process. They also allow teams to identify issues earlier in the development cycle, when they are much easier and cheaper to fix.
Standardized tests make it possible to compare different designs or components objectively. This is especially important when selecting between different modules or suppliers, where performance differences may not become apparent without structured testing.
How can modern measurement technology accelerate development cycles without compromising quality?
Enrico Brinciotti: We’ve established that testing for real-world variables is important, but field testing can be time-consuming and expensive, so how do we achieve this in the lab? Modern measurement technology allows engineers to simulate a wide range of real-world conditions. Teams can recreate scenarios such as weak signals, high interference, or varying network conditions, all within a controlled environment.
This is important because real networks cannot reliably reproduce all conditions, making it difficult to test edge cases consistently. Controlled environments enable repeatable, precise testing across scenarios that would otherwise be impractical to achieve.
This not only speeds up development but also provides much deeper insight into product performance. Engineers can identify root causes quickly, iterate faster, and ensure a higher level of product quality.
When should a team ideally start considering certification testing?
Martin Varga: Ideally, certification requirements should be considered from the very beginning of the development process. Waiting until the end is risky, because failing certification at that stage will mean going back to redesign the product, which is both costly and time-consuming.
Early testing, aligned with certification standards, ensures that potential issues are identified and addressed long before the official certification process, making the final step that much smoother. It also provides confidence that the product will not only meet minimum requirements but perform reliably and effectively in real-world conditions. Having that confidence adds genuine value to the product and/or brand.
If you could give developers only one piece of advice, what most reliably prevents poor testing from resulting in a poor product?
Martin Varga: The most important advice is to test early and test realistically. Don’t rely on simple functional checks or assume that lab conditions reflect the real world.
Instead, treat testing as a way to understand and control performance, rather than using it merely to validate functionality. This is essentially the minimum requirement, and customers naturally expect better than that.
Focus on how your product behaves across a wide range of scenarios and conditions. Even if problems are not immediately visible, they may still exist. Testing comprehensively is the only way to uncover them before customers do.
Enrico Brinciotti: Proper testing is not just a technical requirement; it’s a way to reduce risk, protect your brand, and ensure long-term success. In many cases, it’s a relatively small investment that prevents much larger problems from happening later.
