Lecture 1 (Jan 10):
-
software engineering: barrier between computer science and impact on real world
-
all deadlines on Friday at 6pm
Lecture (Jan 19)
Black Box Testing
Pros - find high severity bugs (from user POV) - easy to get started (more clear what to test, client-facing methods) - less overfitting to implementation: emphasis on specification - parallelize development and testing (no need to access implementation)
Cons - need well-defined specifications - intentional handicapa: can’t use the implementation - poor explanability
how to read a specification to write tests: - input/output equicvalence classes - avoid redundant tests - boundry classes
equivalence classes
-
patterns on types of inputs
-
certain partitions of the input space/output space that has similar behaviour
input/output partitioning
-
partitioning of all the possible inputs to CUT into equivalence classes which characterize sets of inputs with something in common
-
choose representitive to test and if those tests pass all values from that representitive should work
Automated Testing
-
generate test inputs that expose bugs in a program
-
generate test suites that expose bugs in a program
-
concolic + symbolic execution
-
finds violations that are very broad
Fuzz Testing
-
aims to solve the test-input generation problem
-
fuzzing algorithms are test-input generation algorithms where: elements of randomness, use feedback execution or analyze P(i) to guide generation of next input
Less random inputs - generator-based fuzzing - property-based fuzzing - grammar-based fuzzing - mutational fuzzing: start from existing inputs and alter them slightly