How Many Product Samples Do We Really Need To Test For Reliability And Compliance

You need to be aware that during new product development, and also during mass production runs, a number of samples will need to be tested in different ways. The objective is generally to assure their quality, their reliability, and their safety, and to get certified for import and sale.

But how many might be needed? Here’s an example for you based on an electro-mechanical product to be tested…

 

Which samples to test?

During any product development, the need for samples to test is a hot topic. You may be surprised that so many are required and concerned about the costs. That’s a valid concern, but the testing is necessary (I’ve spoken many times about the risks of skipping testing) if you want the peace of mind that your products will be safe and not lead to expensive recalls or legal problems if a consumer gets hurt. Some tests are destructive, so be prepared to foot the bill for some samples that will never be seen again.

Let’s say you have already finished product development, and the product’s design is frozen. The next step is to confirm the reliability, safety and compliance of the product being sold on the market. The samples being tested must be the same as the mass-produced products. That means using the same materials and components, production processes, and production equipment as will be used in mass production.

For this reason, final PP samples should be used for testing. If tooling is necessary, the samples must be made with parts coming from that tooling.

 

What kinds of testing needs to be done to assure product reliability and safety?

Broadly speaking, your new electro-mechanical product (and its packaging) will need to undergo reliability testing and compliance testing.

At Sofeast we help our clients get a bespoke testing plan for their product’s reliability & performance tests as well as compliance & certification tests. In most cases, we consult on the client’s requirements and create the plan; we then do some work internally (when it comes to reliability) or we help find suitable laboratories.

You can see some of the common reliability/durability tests we undertake for clients here, including tests such as HALT testing or package drop tests:

sofeast reliability tests

In the case of product compliance testing, tests required will depend on which region the product is to be sold in and the product type. The number of standards to test to in order for a product to be compliant is almost never-ending because there are so many product types out there. To give you some idea of what’s involved in compliance testing, take a look at this example product, a reusable stainless steel water bottle: Stainless steel bottle testing for safety & compliance. How is it done?

 

An example certification plan and samples required for a consumer electronic product to be sold on the American market

Here’s an example (based on a real case) of the samples needed for the different tests of an electronic device to be imported into the USA to give you an idea of what’s required:

  • 8 units for FCC/CEE (ionizing radiation/energy efficiency)
  • 2 units for ESD (electrostatic discharge)
  • 23 units for various reliability tests
  • 5 additional samples to be held in reserve in case they’re required for reliability testing and ESD.
  • 3 boxes full of finished products for ISTA-2a transportation testing (packaging reliability testing) which will drop, vibrate, and shock the whole box

So that’s a total of 38 samples plus however many are inside three boxes.

 

Are the tests destructive or can the samples/products be used again later?

The FCC/CE tests performed for compliance will not be destructive (there may be some cosmetic damage or marks), however, sometimes the certifying labs may troubleshoot any failures by checking and handling PCBs or other internal components. They usually don’t modify anything, but they may use different loads or filters to check the electronics signals or emissions which may or may not have a destructive effect on the sample product. Therefore for the samples being used for FCC/CE, we recommend full refurbishment by running each product through the rework process to check its cosmetic integrity and functionality per its specifications, and ensure that it passes the final test and final QA before returning to stock or shipping to customers.

The reliability tests and the ESD tests are destructive tests. I recommend not using the parts or products after the tests have been conducted, so they should be scrapped.

ISTA-2a transportation tests may or may not be destructive. This depends on if the packaging selected is durable enough to pass these tests. If it is, the products inside should not be damaged.
Typically the tests are started on just one box of products. If any failures were found on the packaging all the tests would be stopped until those issues are resolved. The one box of units that were tested would be carefully examined to see if any units were damaged as a result of the transportation tests for packaging reliability.
Before they can go back into stock, all units in this box must go through a detailed review first by the reliability test engineer, then the R&D engineering team, and finally by the production to rework and/or refurbish (if needed), and re-certify passing all quality and final tests before they are released for sale.
Any remaining units that were not used for testing can be returned back to stock minus the carton until the issues on the packaging are resolved.

 

Conclusion

Aside from the cost of performing the reliability and compliance testing, you need to take into account the cost of the sample products and packaging to be tested. These are typical NRE costs that importers need to budget for during their new product launch.

When working with a supplier or consultant, they should be able to advise how many samples are needed and for what purposes. They should also provide guidance on whether the samples can be reused in future, and under what circumstances.

About Renaud Anjoran

Our founder and CEO, Renaud Anjoran, is a recognised expert in quality, reliability, and supply chain issues. He is also an ASQ-Certified ‘Quality Engineer’, ‘Reliability Engineer’, and ‘Quality Manager’, and a certified ISO 9001, 13485, and 14001 Lead Auditor.

His key experiences are in electronics, textiles, plastic injection, die casting, eyewear, furniture, oil & gas, and paint.

This entry was posted in New Product Launch and tagged , , , . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *