There are a bunch of tests where this is done incorrectly. There are two correct patterns (one more mobly-ish than the other).

either:

try:
  do command
  assert.fail('unexpected success')
except ChipStackError as e:
  check expected error

or

with asserts.assert_raises(ChipStackError)
  do command and check error

BUT what we have in a bunch of tests is

try:
  do command
except ChipStackError as e:
  check expected error

which will pass the test if the command unexpected succeeds.

This is common enough that we should define a helper function to do it properly AND the tests need a scrub to make sure the incorrect pattern is not used.