Monthly Archives: January 2023

Orthogonal Array Testing Techniques for Automotive Infotainment Validation

Automotive electronics has been undergoing a rapid transformation, much like consumer tech such as smartphones or tablets. More often than not, automotive tech and consumer tech are based on similar chip architectures and technologies.

However, they are still a world apart.

Let us consider the case of your automotive infotainment system.

Unlike your smartphone, the IVI system is subjected to constant G forces and vibrations during its lifecycle. It also must withstand extreme temperatures and humidity. There are several legal and regulatory issues associated with infotainment systems.

Additionally, they are integrated with various other automotive subsystems. So, in case of failures, the safety of the passengers should be ensured.

It is, hence, of great importance that infotainment system development teams perform rigorous testing of these devices before they are integrated with the vehicle.

In this article, we analyse the Orthogonal Array Testing technique, commonly referred to as OAT or OATS. It is a reliable software validation methodology for automotive infotainment testing.

The Need for Orthogonal Array Testing

In today’s automotive software development lifecycles, it is common for the development phase to extend beyond the pre-planned timelines. In such scenarios, the timelines for the testing phase may have to be compromised by a week or so.

One of the most desirable qualities in a tester is his/her ability to be flexible. They should understand that they may have planned for 3-4 weeks of testing, but at the end of an extended development phase, they are only given 1.5 – 2 weeks’ time for testing and validation.

The tester should be able to work within these timelines by considering areas that can be automated, onboarding of additional resources, putting in extra effort, etc.

The tester should also be able to optimize the test suite and come up with effective testing combinations that can uncover the bugs quickly. But this doesn’t mean that the tester can perform random testing (as that will not ensure complete coverage).

This is where the concept of Orthogonal Array Testing comes into the picture.

Using the orthogonal array testing methodology, the tester can optimise the test suite so that it is capable of detecting all types of faults using pairwise combinations of input parameters. This will help in minimizing testing efforts, while also identifying the bugs efficiently.

In this software testing methodology, orthogonal arrays are used to create the test cases. It is a statistical testing concept that is particularly useful when there is a large number of input combinations to be validated for the system under test.

By pairing and combining inputs, the total number of test cases are significantly reduced. And if the right inputs are chosen to be paired together by the tester, the testing coverage and efficiency is also enhanced.

Importance of OAT Automotive Infotainment Testing

An infotainment system receives information from a GPS receiver, GSM receivers, RF receivers, and more. So, it is a multi-engineering topic. It receives various types of inputs and multiple combinations of inputs.

In the input test data for validation of infotainment/entertainment applications like CD drives, the number of file formats itself will be huge. Hence, the validation cannot be done within a timeframe of just 2-3 weeks. Also, it will be very expensive to keep testing all different functionalities, one by one.

One more aspect to consider is that after the development cycle, usually the functional testing phase will be on aggressive timelines. So, in a limited timeframe, it is important to test maximum features and uncover the bugs efficiently.

Orthogonal Array Testing During Unit Testing Phase

At Embitel, we perform orthogonal array testing for infotainment systems, right from the unit testing stage.

In unit testing, it is referred to as MCDC (Multiple Condition Decision Coverage). If we do not perform MCDC, we will end up writing a lot of test cases.

In unit testing, as per MCDC, the developer checks the different conditions or input combinations for making a decision. In automotive software validation, it is crucial to check a lot of combinations before we make any decision.

Let’s explore this testing methodology with an example.

OAT can be used in all types of industries, any application, during any phase of testing, any levels or types of testing.

In unit testing, consider that we are validating different combinations of 5 variables. Each variable has 5 different states.

And so on….

Total no of combinations = 5^5 = 3125 combinations.

After applying Orthogonal Array Testing methodology, the total number of test cases can be reduced to 33.

OATS will remove all the duplicated pairings and provide an efficient list of test combinations as shown below:

This tutorial on orthogonal array testing helps you to understand how to create the orthogonal array for a test scenario.

OATS for System Testing of Automotive Infotainment Software

Let us consider the testing process of infotainment system to check whether there is a sudden increase in volume when the audio parameters change.

Audio Parameters:

1. Bass
2. Treble
3. Loudness
4. 7 bands of equalizers

Total number of combinations without OATS:

(Bass=2)*(Treble=2)*(Loudness=2)*( 7 Equalizers=2 ^7) = 2048.

After applying orthogonal array testing methodology, the following 10 test cases are found to be sufficient to ensure good testing coverage:

Advantages of Orthogonal Array Testing

• OAT method can be used in the earlier testing stages itself. This implies that right from unit testing, this method can be used for evaluation.
• OATS reduces time with increased coverage of test cases (due to combinations/pairings).
• It is best suited for testing applications in which there is a large amount of test data/requirements to validate. For eg., Automotive infotainment testing.
• OATS is an efficient and effective testing process.

Summary

As part of the role of a software tester, especially when performing black box testing, the motive is to try and break the code in every possible way. The system is usually tested from the end-user’s point of view.

Even if the testing is done for a small component of a larger system, it is observed that defects/bugs are found easily when the test data is combined/paired. This is the basis for the orthogonal array testing methodology.

While testing using OATS, it is important to create test cases by pairing the right test inputs and combinations. This will help in uncovering bugs as early as possible, with minimal effort and time investment.

Author Bio

Yogesha Lakkanna, Associate Director – Embitel

Yogesha Lakkanna is responsible for Verification and Validation at Embitel. He is an Electronics and Communication Engineer and Certified Testing Professional. He has 20+ years of experience in Release Management, Quality Assurance and Test Lifecycle management in the field of Realtime Embedded Applications in Avionics, Consumer, Medical and Automotive Electronics.

Yogesha is also interested in Farming, Literature, Music, and Social Service. He is an active member and volunteer in various cultural and social organisations in Bangalore.

How to Build Better Automotive Applications with Model-in-Loop Test Approach

“In a highly competitive and time-sensitive market like automotive, it is crucial to test new solutions before they are fully realized. Model-in-loop test makes this possible by creating mathematical models that represent the solution in a part-real and part-virtual manner.”

It’s a busy day for the automotive team of a tier-1 company. Engineers are rushing to complete the modules assigned to them so that the unit testing can be performed. The engineers are worried about multiple testing iterations that would follow the module development. The deadline is fast approaching. It is a moment of calm before the project manager goes into a tizzy and starts questioning the team about the delay. Oh.. that’s a messy situation to be in!

One good decision at the beginning of the project and all of this could have been avoided. If the automotive team had taken the model-based design approach, the situation would be much different.

Model-in-loop testing would start with models that would virtually represent each unit under test. Early detection of bugs and errors would be possible to ensure a bug-free solution and developers would be spared from the pains of manual coding, a win-win for all stakeholders.

What is Model-in-Loop Testing?

Model-in-loop testing is a technique to represent the behaviour of an embedded system in the form of mathematical models. These models are used to verify the system or the sub-system.

Testing automotive embedded software at the model-in-loop level is the first step in the process of V&V. It implies that the controller and the hardware are simulated using models in a simulation environment. And that the controller model is able to control the plant model (hardware) as per the specified requirements.

Such testing helps in verifying the software very early and bugs can be identified before they creep into the program. Needless to mention that rectifying errors at a model level is inexpensive compared to a fully realized ECU.

MIL test proves to be very effective when it is performed along with the process of model creation or shortly after that. Since MIL is a requirement-based test, the test cases are derived directly from the requirements for the automotive software under test.

One of the reasons why model in loop test has been adopted so widely in the automotive industry is the availability of simulation tools and environment such as SIMULINK. Designing the plant model, controller model and the complete test harness have been simplified to a large extent. The much-needed traceability is provided by these tools and hence, they are also recommended by standards such as ISO 26262 and ISO 21434 (cybersecurity).

No amount of theoretical explanation can actually get you the real picture of how MIL testing is performed and what impact it does have on the development lifecycle. So, lets take a deep dive into understanding the MIL test setup for testing a real-world automotive application.

Model-in-Loop in Action- MIL Test Setup for an Automotive Lighting Module

Automotive lighting modules are used for safety-critical applications to display faults or failure. This requires lighting systems to be fail-safe as well. Therefore, model-based approach works best for development of such modules.

We will try to understand the MIL test setup for the development of an automotive lighting module. Right from requirement elicitation and test case creation to building the test harness and executing the tests, let’s witness MIL test in action.

Solution Under Development- An Automotive Lighting Module for a Headlamp

Inputs to be considered while performing MIL test for this automotive lighting solution is ignition, switch, and intensity.

The outputs are left lamp and right lamp.

Requirement– If the ignition is ‘on’ and if switch is ‘on’, the lamp has to be turned on.

Step-1

Intensity 0 to 100% based on the input given by the user using a button or through infotainment system.

Based on these requirements, the test strategy will be designed. Let’s understand the test strategy for the requirements mentioned above.

• Initially, the switch is ‘off’.
• It is now turned-> on; now we check the intensity of the lamp for a duration of 100 seconds (can be changed)
• Now, in order to test the intensity for 0-100 seconds, we have to give the inputs from 0-100 seconds.
• In the test description, the pre-conditions and the expected results are already logged. Since, we are required to test both the true and false condition, we will test with both ignition ‘on’ and ignition ‘off’ scenario. As have taken 100 seconds as the testing time, we will keep the ignition ‘on’ for first 50 seconds and ignition ‘off’ for the next 50 seconds.

Step-2

The entire testing pre-conditions and inputs that we described above are achieved by a signal builder block.

This block creates the corresponding test data usually from an excel sheet (test vector sheet) containing all the relevant data. The data points required for the current MIL test are- time, ignition status, switch status and intensity.

Step-3

There is one more column in the test vector sheet which is the expected result. During the test, the output result will be compared to the expected result and hence, test pass/fail will be evaluated.

Step-4

The data from the test vector sheet can be imported to the signal builder block with the help of a few clicks on SIMILINK tool. After the import, the data from the test vector sheet is displayed as a wave form.

For the current test, we get three wave form graphs-

• Time and ignition status
• Time and intensity
• Time and switch status

Step-5

Now that the signal builder block is ready, we need to connect it to the control model which virtually represents the automotive ECU with the lighting module application.

But before we get ready to test the signals, we must first log the expected result to which the actual output will be compared.

We will use signal logging option in the SIMULINK tool to log the expected results. To do so, we must carry on one simulation using the same inputs and log the expected result. Logging for expected result will now be stopped and logging for model signals will begin. The simulation will be done once again. The point to be noted here is that logging of both expected result and actual result from the model must be in the same name.

Step-6

Now that we have both expected results and the actual output from the model, we can now compare and inspect whether the test failed or passed.

Using data inspector tool, we can compare both results. An html report is generated upon comparing that clearly shows the pass/fail status.

Conclusion

Automotive OEMs and tier-1s are always looking to beat the clock when it comes to building innovative automotive solutions. After all, these solutions become the USPs in an industry as competitive as automotive.

Model based design approach, along with Model-in-Loop testing, play a crucial role in enhancing the efficiency of automotive software development and validation.