Automated Tests

Our automated testing suite uses sbt and the Scala Test FunSuite, just like the NetLogo/NetLogo project. These tests ensure each Models Library model is following our style guide, correctly compiles in the newest version of NetLogo, and more.

As of e35ce19, the below testing suites are utilized:

1. NLWTests

• All primitives are supported by NLW (only run if org.nlogo.onLocal is set due to false positives)

2. SpellCheckTests

• Models must not contain typos

3. InfoTabUrlTests

• URLs used in info tabs should be valid (only run if org.nlogo.onLocal is set due to false positives)

4. PreviewImagesTests

• Models should have manual previews only if needed or permitted
• Models should have committed preview iif they're 3d or require manual preview
• Images should be in .gitignore iif they don't need manual previews
• .gitignore should not have duplicates
• .gitignore should not contain preview entries for non-existing models

5. PreviewImagesTests:

• Models should have manual previews only if needed or permitted
• Models should have committed preview iif they're 3d or require manual preview
• Images should be in .gitignore iif they don't need manual previews
• .gitignore should not have duplicates
• .gitignore should not contain preview entries for non-existing models

6. ButtonTests:

• Buttons should be disabled until ticks start if they trigger a runtime error
• If any button is disabled until ticks start, ensure at least one enabled button that resets ticks

7. BehaviorSpaceTests:

• BehaviorSpace experiment names should not start with "experiment"

8. NameTests:

• There should be no duplicate model names

9. BadInterfacesTests:

• Widget sections should have proper lengths
• Output widgets should have reasonably sized font
• Textboxes (i.e., notes) should be wide enough

10. InfoTabsTests:

• All models' info tabs should have WHAT IS IT? section
• Most info tabs should start with WHAT IS IT? section
• Length of first paragraph of WHAT IS IT should be >= 40 and <= 700
• First paragraph of WHAT IS IT should be unique
• Info tabs should not have empty sections
• Info tabs should not have repeated sections
• Bullet list using dashes should have space after dash
• Info tabs should not contain reviewer comments (e.g. {{{ or }}})
• Info tabs should not contain HTML tags (other than <sup> and <sub>)
• Code blocks should only include NetLogo code

11. UpdateModeTests:

• Models should use tick-based updates unless otherwise specified

12. LegalInformationTests:

• The last line of the info tab should be a well-formatted legal snippet inside an HTML comment
• The notarizer should work properly on all models
• Model file should be identical to output of Notarizer

13. ModelFileTests:

• Models should not use tab characters anywhere
• Models should not countain trailing whitespace

14. CrossReferenceTests:

• all cross-referenced single models exist
• all cross-referenced directories exist and are directories

15. ModelCompilationTests:

• Compilation output should satisfy various properties

16. VersionTests:

• All models are readable
• Version should be one of NetLogo 6.1.1

17. ViewTests:

• Models should have integer patch sizes
• Most models should have a frame rate of 30
• Saved view size must match size computed from the saved patch size and screen-edge-x/y

18. MergeConflictTests:

• No merge conflicts appear in model files

19. CodeTests:

• Forbidden primitives are not used
• Plot names should be case insensitive
• Plot-pen names should be case insensitive
• All identifiers should be lowercase
• Plotting commands should not be in the main code
• Lines should not be longer than 170 characters
• Anonymous procedures with zero or one argument should not use brackets
• Code should not contain reviewer comments (e.g. {{{ or }}})

This is a total of 19 testing suites (only 17 of which are run by default).

Running the automated test suite

We'll add more details on how to set up sbt at a later date.

Run NetLogo models

sbt test

Run NetLogo 3D models

sbt -Dorg.nlogo.is3d=true

Run All tests (only Locally)

As of f9ab7c5, this is accomplished similarly to how we run 3D models tests:

sbt -Dorg.nlogo.onLocal=true
sbt -Dorg.nlogo.onLocal=true -Dorg.nlogo.is3d=true

Build Server

When you publish a branch on the NetLogo/models repo, that will also automatically trigger a build on our build server, Jenkins.

If you need an account, please reach out to the person at the CCL currently in charge of Jenkins.