By John Brunstein
25 Jan 2016
“Quality” is one of those words that gets used a lot – particularly in association with any commercial product. After all, if you’re going to pay for something, you want a “quality” product in return, not something poorly made or not living up to its advertised performance or specifications. Anyone selling a product knows this, so regardless of what you go to buy, from whom, they all will tell you it’s a “quality product” or “made with quality”, or something similar. The difficulty is that these terms are self applied, they’re overused, and in many cases, they are outright wrong. Few if any readers of this will not be able to think from their own experience of examples of things which were presented with the word “quality” somehow attached, but somehow having lost the critical modifier “poor”!
One consequence from this is that consumers have rightfully become very cynical about the term “quality” as applied to just about anything offered to them. How can they know that a claim to quality in some product is actually legitimate, and not just part of slick advertising? One way is to find, or try to find, unbiased, third party reviews of the product or service in question. This can be a challenge however as what appear to be unbiased and third party reviews – especially those available online – could be paid for “plants” by the supplier of the service or product. Again, savvy consumers can often spot these if they’re too overt, but skillfully done fake reviews can be hard to detect. Another problem with that sort of assessment of “quality” is that in general, people are more likely to provide negative feedback than positive feedback; people frustrated when something doesn’t work as advertised appreciate the chance to vent their frustration while satisfied customers are too busy being satisfied to take the time to make reviews. In order to put a meaningful basis behind the application of the word “quality” to their products and / or services, industrial scale processes have developed the concept of a “Quality System”. The basic idea behind any quality system is:
• to have good definitions of what constitutes quality in a product or service;
• to have a way to measure this product quality;
• to have a demonstrable, repeatable way of achieving this quality; and
• to have a way of convincingly demonstrating this entire process to the end customer.
Let’s examine each of these in turn, to see how they work together to give actual meaning to the use of the word “quality”.
What constitutes Quality?
First, you need to have definitions of what constitutes quality in the context of your product or service. These can include physical attributes such as size, shape, colour, density, texture, colour, flavour, scent, hardness, waterproofness, ability to dissolve in ____, strength of magnetization… the list is quite literally endless, because there’s any number of possible attributes or combination of attributes we can imagine. However, for any given product intended for a specific purpose, only a specific limited subset of these will make sense. Let’s take a simple example – a pencil eraser. If you try to imagine what makes a quality pencil eraser, most readers will probably come up with things like size (if it’s the size of a car, or of a sesame seed, it’s not going to be very useful); shape (you want to be able to grasp it); hardness (too soft, and it will mush all over the paper; too hard, and it will scrape through the paper); and functionality (does it remove pencil marks from the paper?) Other attributes are less likely to be critical in describing the quality of an eraser – for instance its colour (they’re not all that off pink colour; some are white, or various gummy-bear-ish shades of translucent). What about, say, magnetism? I don’t think I have ever seen a magnetic eraser, but what if someone marketed an eraser which was magnetic so you could leave it stuck to your fridge door, close at hand when you want to modify your penciled down grocery list? Regardless of whether that sounds like a good product idea, if that was the idea, then magnetism of the eraser would become a meaningful aspect of its quality – its ability to perform as advertised. Finally, there are some attributes we’d probably all agree just don’t make sense for measuring the quality of an eraser – like its taste. It’s not intended as a food item, so this is pretty much irrelevant. The point of our example eraser is that the constituents of quality are very much dependant on what the product or service is, and they really should be dictated by the needs of the end user. For any product or service in a quality system, then, we have to start by assessing what it’s used for, and what aspects of that use are the important quality aspects.
How do we measure Quality?
Now that we’ve defined our quality aspects for a product or service, a Quality System needs to have some way – ideally, an objectively measureable way, otherwise known as a “metric” – to asses each quality aspect against some scale. In addition to some quantifiable measurements of each aspect of quality, we also need to have some acceptable range of values on each measurement. These are known as the “acceptance criteria” for the attribute, and in a Quality System, they’re a crucial part of determining whether each item has acceptable quality. Generally, an item (or process) would have to be tested for, and fall within the acceptance criteria of, every single test or measurement which would each correspond to one of the defined quality aspects.
This gets a bit more complicated than it sounds at first, because for this to be actually objectively reliable, we have to have some sort of proof or evidence that the measurements we take for each metric are themselves accurate. That means we need to have some sort of validation of the test process (which is a proof the test works, and that the measurements are a valid way to assess the intended quality aspect). We also need some sort of calibration samples or controls, which have an independent verification of how they should measure in our test system, and we should run through our test either in parallel with real samples, or at least on a regular interval to prove that our test still works correctly. Since one goal of a Quality System is to be able to demonstrate to others that we are doing these quality tests accurately, we also need to keep detailed records of every step of this process – from how we chose the quality aspects, to how we validated the tests, to the ongoing proof of test functionality.
Another consideration for Quality Systems at this stage is often “we make thousands of widgets each production run, do we need to test every single one or can we test a subset?” The answer to this question depends to some extent on the criticality of the measurement. If it’s our erasers of the example above, if we somehow miss an eraser which fails one acceptance criteria, it’s not a serious outcome. Nobody dies, they just get a not-so-good pencil eraser. However, if we’re making a medical device like part of an artificial heart, then yes, someone does potentially die if a part doesn’t meet acceptance criteria. To address this, there’s usually some form of Risk Assessment process, which asks in formalized terms, “how serious is the potential damage of a product being released that would have failed if tested?” Based on this (recorded, on file) process, we can judge whether every widget needs to be tested individually, or whether it’s OK to test just a statistically relevant portion of each production run. If we decide a subset is ok, then we have to use (and record) accepted statistical methods to show how many widgets to sample to have an acceptable confidence that all widgets in the lot are ok. Once we’ve done all that, we can test our acceptable size sample set, and apply statistics to make supportable calculations as to whether all widgets in the production run are OK, or not. If the statistics say more than a certain % aren’t likely ok (this is yet another acceptance criteria), then we have to either reject the entire lot, or default to testing each widget individually and only accepting the passing ones. If you suspect by now that we’re also going to have to keep and maintain detailed records of all of this, to prove that we correctly assessed the widget lot quality, you’d be right. You’d also have likely come to realize something else inherent to Quality Systems – that is, they require comprehensive and detailed record keeping to be able to not just say things are “quality”, but prove it, each and every step of the way.
How do we maintain Quality?
Now we move on to the third part – having a way to uniformly produce the product or service, such that it will meet our various acceptance criteria. The incentive for this part of the process is purely monetary on the part of the producer; after all, they now have acceptance criteria to meet, and if product is made which doesn’t meet these tolerances, it’s either discarded or subject to reworking to achieve a pass. Either way is expensive and companies don’t like wasting money that way – so they’re strongly incentivized to develop processes to have things made uniformly in a way which maximizes the likelihood of producing in-spec product. Usually, this consists of developing exactly repeatable ways of doing all production processes and defining these in what are called Standard Operating Procedures (commonly just referred to as “SOPs”). These are essentially glorified recipes with a lot of detail, intended to be followed point by point in sequence without any deviation. SOPs however are only useful if they’re followed, so companies with Quality Systems generally also establish comprehensive employee training programs. These ensure that every employee who is supposed to carry out a particular SOP is appropriately trained on how to do it.
An additional feature of this part of a Quality System is a way to document each of the SOPs (and do so in a way which allows for them to be updated or revised, in a controlled manner), which is a process usually referred to as Document Control. In addition to recording SOPs, this also incorporates systems to ensure that only ‘current in use’ versions of SOPs can be accessed by staff. Generally, all employee training would also be extensively documented.
How do we Demonstrate Quality?
Finally, we’re on to the fourth stage of a Quality System: having a way to demonstrate all of the preceding material to external parties such as customers, potential customers, business partners, or regulatory bodies. This is where all of the record keeping from the prior three stages comes together, making up a traceable record of every stage of the production process – from the selection and acceptance of raw materials, to the SOPs followed in turning those into product, the training records of the people who did each part of the production, the validation of test methods used to evaluate final product, and the actual final product test results showing acceptable lot Quality Control (“QC”) values.
You might be inclined to wonder why an unscrupulous company, looking to appear like they had a Quality System, but not willing to put in very considerable actual time and effort to actually have one, can’t just issue some made up QC certificates and show those. The bad news is they can, and some do. Make no mistake about it, it’s a caveat emptor world, and a wise consumer will want to ask a few more questions about what’s behind a quality statement. Companies which actually have a Quality System are more than happy to provide detailed information about it; they understand its value is in assuring the end customer that things are done right. Companies which just have window dressing can only reply with vague assurances, and if pressed for even simple details won’t be able to provide them or will be clearly missing one or more of the four general steps described here.
Another widely used strategy in demonstrating an effective Quality System to the outside world, is through the use of unaffiliated third-party Quality System audits and certificates. If you’ve ever heard of things like “ISO 9001”, “ISO 13485”, “GMP certification”, or similar, that’s exactly what those are. Each of those is a slightly different set of criteria (often, specialized towards a particular industry; ISO 13485 for instance is Medical Test Devices). A company can apply to be audited or reviewed by one of these outside accreditors against a particular inspection checklist, and if the review is successful, receive a certificate. This certificate is a “seal of approval” so to speak from a disinterested third party, which has examined all of the Quality System components in a company and finds evidence that all aspects work together to support a claim of product quality. In non-regulated industries, getting these Quality Systems certifications is optional, and it’s an expensive and time consuming process but many companies actually do it to help assure customers of their commitment to verifiable quality – these companies will usually prominently display their “ISO Certified” status, or similar. In regulated industries different forms of external quality audit – either to third party certificates, or to the regulatory bodies themselves – are mandatory. For example, with Segra Biogen’s natural health products, all stages of Manufacturing, Packaging, Labeling, and Distribution are audited by Health Canada and must be certified to meet GMP (“Good Manufacturing Practices”).
So the next time you see a product advertising its “quality”, stop and examine it – is there a Quality System backing up those claims so you can actually trust them, or is this “Kwality” – and which do you want to spend your hard earned dollars on?