Test Case Factory and Builder

Simple Story

A filled-in checklist is a wish list. Something has to turn it into checks the teacher can work through.

This is that something: it reads the finished checklist and builds the exact set of security test cases it calls for.

Audience: IT-Education experts with no security background. Scope: All classes inside de.tum.cit.ase.ares.api.securitytest — the abstract factory/builder, the Java-specific factory, and the creator, essentialModel, executer, writer, projectScanner, and specific sub-packages. Ares Version: 2.1.5

Related documentation:

• Security Policy Manual — how to write a security policy YAML file
• Security Policy Reader and Director Manual — how the policy is read, directed, and handed to this factory
• Precompile or Postcompile — choosing a mode, and the walkthrough for your build tool

---

1 Prerequisites

• Java 17 or later
• Gradle or Maven 3.8+ for building, with versions compatible with the AspectJ and test plugins used by the project
• JUnit 5 (Jupiter) for test execution
• Ares 2

---

2 Purpose — What Problem Does This Solve?

The Security Policy Reader and Director reads a YAML policy file and selects the correct toolchain. But the actual generation, serialisation, and execution of the security test cases happens in this package.

Given a parsed SecurityPolicy, this package must:

1. Discover which packages, classes, and build tools the student project uses — either from the policy or by scanning the project directory.
2. Load essential data — the list of packages and classes that Ares itself needs at runtime and that must never be blocked by the security policy.
3. Create both architecture test cases (static bytecode checks via ArchUnit or WALA) and aspect-oriented programming (AOP) test cases, which intercept at runtime through AspectJ or ByteBuddy Instrumentation.
4. Write the generated test-case source files to disk.
5. Execute the test cases — first the static architecture checks, then the dynamic AOP enforcement.

Without this package, Ares would know what to enforce but could not turn that knowledge into runnable tests. The next section describes how the package is organised to fulfil this responsibility.

---

3 Architecture Overview

The package is structured around five sub-packages, each handling one step of the pipeline. The table below summarises the design patterns used — understanding these patterns is not required to use Ares.

Click to expand: Design Pattern Reference

Pattern	Where it is used	Why
Abstract Factory + Builder	TestCaseAbstractFactoryAndBuilder → JavaTestCaseFactoryAndBuilder	The factory must produce different kinds of test artefacts (architecture tests, AOP tests, Phobos tests) for different toolchain combinations (Maven/Gradle × ArchUnit/WALA × AspectJ/Instrumentation). The Abstract Factory hides the concrete toolchain behind a uniform interface, while the Builder allows step-by-step configuration of the many required parameters (modes, paths, collaborators, policy).
Template Method	Constructor of TestCaseAbstractFactoryAndBuilder	The test-case creation lifecycle always follows the same steps: inject tools → resolve modes → load essential data → extract policy configuration → guard the supervised package → create test cases. The abstract base class fixes this sequence while letting the Java-specific subclass provide concrete implementations of writeTestCases() and executeTestCases().
Strategy	Creator, Writer, Executer, ProjectScanner, EssentialDataReader (all interfaces)	Each step of the pipeline (creating, writing, executing, scanning, reading) must be independently swappable for different programming languages or frameworks. Defining each step as an interface with a Java-specific implementation allows future language support (e.g., Python) without modifying the abstract factory.
Builder	JavaTestCaseFactoryAndBuilder.Builder, EssentialClasses.Builder, EssentialPackages.Builder	The factory requires 12 parameters (5 collaborators, 2 essential-data paths, 3 modes, 1 policy, 1 project path). A builder with named setter methods prevents parameter-ordering mistakes, enforces mandatory fields via Objects.requireNonNull in build(), and makes the construction code self-documenting.
Caching (Memoisation)	JavaCreator.cacheResult(classPath, supplier)	Building call graphs (WALA) and importing Java classes (ArchUnit) from compiled bytecode is computationally expensive. A ConcurrentHashMap-based cache — deliberately an instance field, since a JavaCreator lives for exactly one factory build — ensures that these operations are performed at most once per factory build, even when that build processes multiple test cases, without leaking one run's analysis to a later run in a long-lived Java Virtual Machine (JVM).
Immutable Value Objects (Java Records)	EssentialClasses, EssentialPackages	The lists of essential packages and classes must not be accidentally modified after parsing — a mutated list could silently weaken security enforcement. Java Records guarantee immutability and provide automatic equals(), hashCode(), and toString().
Annotation-Driven Configuration	@StudentCompiledClassesPath + PathLocationProvider	The location of compiled student classes varies between Learning Management Systems. Rather than hard-coding paths, an annotation lets instructors declare the path on their test class, and ArchUnit's LocationProvider SPI reads it at runtime — fully decoupling the test framework from the deployment environment.

Package structure:

securitytest/
├── TestCaseAbstractFactoryAndBuilder.java    ← abstract base (Template Method)
├── ReservedPackageGuard.java                 ← rejects supervised packages under trusted prefixes
└── java/
    ├── JavaTestCaseFactoryAndBuilder.java     ← concrete factory (Builder)
    ├── StudentCompiledClassesPath.java        ← annotation
    ├── creator/       ← generates test-case objects from the policy
    ├── essentialModel/ ← reads essential (always-allowed) packages/classes
    ├── executer/      ← runs architecture + AOP tests at runtime
    ├── projectScanner/ ← auto-detects build tool, packages, main class
    ├── specific/      ← ArchUnit LocationProvider integration
    └── writer/        ← serialises generated test sources to disk

---

4 The Abstract Factory and Builder — TestCaseAbstractFactoryAndBuilder

This is the central orchestration class of the entire package. Its constructor acts as a Template Method: it fixes the exact sequence of steps required to set up security test cases, while its abstract methods (writeTestCases, executeTestCases) let subclasses provide language-specific logic.

4.1 Attributes

The class holds five groups of attributes:

Tool Collaborators (injected, all @Nonnull):

Attribute	Type	Responsibility
creator	Creator	Generates test-case objects from the policy
writer	Writer	Serialises generated test sources to disk
executer	Executer	Runs architecture and AOP tests
essentialDataReader	EssentialDataReader	Reads essential packages/classes from YAML
projectScanner	ProjectScanner	Auto-detects build tool, packages, main class

Modes and Paths (resolved at construction time):

Attribute	Type	Source
buildMode	BuildMode	From policy, or auto-detected by projectScanner.scanForBuildMode()
architectureMode	ArchitectureMode	From policy, or default WALA
aopMode	AOPMode	From policy, or default INSTRUMENTATION
projectPath	Path (nullable)	Injected — where the student project lives

Essential Data (loaded from YAML at construction time):

Attribute	Type	Content
essentialPackagesPath	Path	File system path to EssentialPackages.yaml
essentialClassesPath	Path	File system path to EssentialClasses.yaml
essentialPackages	List<String>	Aggregated list of all essential packages
essentialClasses	List<String>	Aggregated list of all essential classes

Policy-Derived Configuration (pinned by the SecurityPolicy, or scanned when no policy is present):

Attribute	Type	Meaning
testClasses	List<String>	FQCNs of the instructor's test classes — never restricted
packageName	String	The student's main package — subject to the security policy
mainClassInPackageName	String	The student's main class (e.g., Main)
resourceAccesses	ResourceAccesses	Which I/O operations the student is allowed to perform

Generated Test-Case Lists (populated by the creator during construction):

Attribute	Type	Contains
architectureTestCases	List<ArchitectureTestCase>	Static bytecode analysis rules (ArchUnit / WALA)
aopTestCases	List<AOPTestCase>	Runtime interception configurations (AspectJ / Instrumentation)
phobosTestCases	List<PhobosTestCase>	Phobos framework test cases

4.2 Constructor — The Template Method

The constructor executes a fixed sequence of six steps. This sequence is the same regardless of the programming language or toolchain:

Step 1 — Inject tools
  ├── Objects.requireNonNull for all 5 collaborators

Step 2 — Resolve modes
  ├── buildMode         ← explicit value || projectScanner.scanForBuildMode()
  ├── architectureMode  ← explicit value || default WALA
  └── aopMode           ← explicit value || default INSTRUMENTATION

Step 3 — Load essential data
  ├── essentialPackages ← essentialDataReader.readEssentialPackagesFrom(path)
  │                        .getEssentialPackages()
  └── essentialClasses  ← essentialDataReader.readEssentialClassesFrom(path)
                           .getEssentialClasses()

Step 4 — Extract configuration (fail-closed when a policy is present)
  ├── Policy present:
  │     ├── null SupervisedCode          → SecurityException
  │     │                                  ("security.policy.supervised.code.required")
  │     ├── packageName                  ← policy value; null/blank → SecurityException
  │     │                                  ("security.policy.supervised.package.required")
  │     ├── mainClassInPackageName       ← policy value || projectScanner
  │     │                                  .scanForMainClassInPackage()  (only fallback)
  │     ├── resourceAccesses             ← policy.resourceAccesses
  │     └── testClasses                  ← ONLY policy.testClasses (never scanned)
  └── No policy (legacy scan path):
        ├── packageName                  ← projectScanner.scanForPackageName()
        ├── mainClassInPackageName       ← projectScanner.scanForMainClassInPackage()
        ├── resourceAccesses             ← ResourceAccesses.createRestrictive()
        └── testClasses                  ← projectScanner.scanForTestClasses()

Step 5 — Guard the supervised package
  └── ReservedPackageGuard.validatePackage(packageName)
        → SecurityException if the package falls under a trusted
          infrastructure prefix

Step 6 — Create test cases
  └── creator.createTestCases(...)  ← fills architectureTestCases,
                                       aopTestCases, phobosTestCases

Step 5 exists because the static analysers and the runtime AOP treat certain package prefixes (Ares internals, WALA's infrastructure prefixes) as trusted — a student submission declaring such a package would be trusted by name and bypass every check. ReservedPackageGuard therefore fails closed before any analysis or execution. The same guard is applied again later: JavaCreator.createTestCases validates the class names in the imported bytecode via ReservedPackageGuard.validateClassNames(...) (catching precompiled or generated classes that never appear in the scanned source), and JavaProjectScanner.scanForPackageName() filters reserved prefixes out of its candidates.

4.3 Policy vs. Scanner Fallback

The constructor distinguishes two modes, depending on whether a SecurityPolicy was supplied:

Mode 1 — Policy present (fail-closed): A present policy must authoritatively determine the enforcement scope. Missing or blank values are not silently filled in by scanning the student-controlled project — the constructor refuses to run instead:

Policy field	Behaviour
regardingTheSupervisedCode is null	SecurityException (security.policy.supervised.code.required) — no scanner fallback
theSupervisedCodeUsesTheFollowingPackage is null/blank	SecurityException (security.policy.supervised.package.required) — no scanner fallback
theMainClassInsideThisPackageIs is null/blank	Falls back to projectScanner.scanForMainClassInPackage() — the only remaining scanner fallback, safe because the main class is used only for code generation, not as an enforcement boundary
theFollowingClassesAreTestClasses	Used as-is — test classes come only from the policy. Deriving the exempt set from the project would let a student add an @Test class to obtain a blanket exemption from every check
theFollowingResourceAccessesArePermitted	Used as-is from the policy

Mode 2 — No policy (legacy scan path): Without a policy, the scanner auto-detects everything: packageName via scanForPackageName(), mainClassInPackageName via scanForMainClassInPackage(), testClasses via scanForTestClasses(), and resourceAccesses is set to the fully restrictive ResourceAccesses.createRestrictive().

This design allows Ares to work without a policy file, while a present policy pins the enforcement scope instead of merely taking precedence over the scanner.

4.4 Abstract Methods

Method	Signature	Responsibility
writeTestCases	List<Path> writeTestCases(Path testFolderPath)	Serialises the generated test cases to files in the given directory. Returns the list of created file paths.
executeTestCases	void executeTestCases()	Runs the generated architecture tests (static analysis) and AOP tests (agent configuration).

---

5 The Java Factory — JavaTestCaseFactoryAndBuilder

This is the concrete implementation of TestCaseAbstractFactoryAndBuilder for the Java programming language. It delegates all work to the injected collaborators after casting the generic test-case lists to Java-specific types.

5.1 Write and Execute

writeTestCases(Path testFolderPath): Casts the generic lists and delegates:

writer.writeTestCases(buildMode, architectureMode, aopMode,
    essentialPackages, essentialClasses, testClasses,
    packageName, mainClassInPackageName,
    architectureTestCases.stream()
        .map(tc -> (JavaArchitectureTestCase) tc).toList(),
    aopTestCases.stream()
        .map(tc -> (JavaAOPTestCase) tc).toList(),
    phobosTestCases.stream()
        .map(tc -> (JavaPhobosTestCase) tc).toList(),
    testFolderPath);

executeTestCases(): Same casting approach, delegates to executer.executeTestCases(...).

5.2 The Builder API

The factory is constructed via a fluent builder with 12 parameters:

JavaTestCaseFactoryAndBuilder.builder()
    .creator(new JavaCreator())
    .writer(new JavaWriter())
    .executer(new JavaExecuter())
    .essentialDataReader(new EssentialDataYAMLReader())
    .projectScanner(new JavaProjectScanner())
    .essentialPackagesPath(Path.of("EssentialPackages.yaml"))
    .essentialClassesPath(Path.of("EssentialClasses.yaml"))
    .buildMode(BuildMode.GRADLE)
    .architectureMode(ArchitectureMode.WALA)
    .aopMode(AOPMode.INSTRUMENTATION)
    .securityPolicy(policy)
    .projectPath(Path.of("/path/to/project"))
    .build();

The first five parameters (creator, writer, executer, essentialDataReader, projectScanner) plus the two essential-data paths (essentialPackagesPath, essentialClassesPath) are mandatory — build() throws NullPointerException if any is missing. The remaining five parameters (buildMode, architectureMode, aopMode, securityPolicy, projectPath) are optional — they fall back to scanner defaults or safe defaults.

Note: In practice, instructors do not call this builder directly. The SecurityPolicyJavaDirector (see Reader and Director Manual) constructs the factory automatically based on the policy file.

---

6 Creating Test Cases — The creator Package

6.1 Creator Interface

Aspect	Detail
Role	Defines the strategy interface for generating test-case objects from a parsed security policy. Implementations populate three lists: architectureTestCases, aopTestCases, and phobosTestCases.
Pattern	Strategy Pattern. The interface allows different implementations per programming language (currently JavaCreator, potentially PythonCreator in the future).
Key method	createTestCases(BuildMode, ArchitectureMode, AOPMode, List<String> essentialPackages, List<String> essentialClasses, List<String> testClasses, String packageName, String mainClassInPackageName, List<ArchitectureTestCase>, List<AOPTestCase>, List<PhobosTestCase>, ResourceAccesses, Path projectPath) — populates the provided test-case lists in place.

6.2 JavaCreator — The Core Test-Case Generator

Aspect	Detail
Implements	Creator
Role	The heart of this package. Extracts bytecode metadata, computes allowed packages/classes, and generates both fixed (always-on) and variable (policy-dependent) test cases for Java projects.
Pattern	Strategy (implements Creator), plus internal use of Caching (Memoisation) to avoid recomputing expensive call graphs.

What it does step by step:

Each invocation proceeds through two phases — extraction (reading class data from disk) and preparation (computing permissions) — before generating the final test-case objects:

1. Extraction (cached):

   • Computes the classPath via BuildMode.getClasspath(projectPath, packageName). This is more than picking target/classes (Maven) vs build/classes/java/main (Gradle): the method appends the package path to the build directory and interprets withinPath-style prefixes such as classes/java/main/... or classes/... by rewriting them onto the actual build directory.
   • Validates the imported bytecode via ReservedPackageGuard.validateClassNames(...) — any compiled class declared under a trusted infrastructure prefix aborts test-case creation with a SecurityException.
   • Imports JavaClasses via the ArchitectureMode (ArchUnit's ClassFileImporter).
   • Obtains the CallGraph as a lazy Supplier via the ArchitectureMode — it is never eagerly built here (null for ArchUnit, WALA's CustomCallgraphBuilder for WALA mode). Only the classPath and the JavaClasses are computed eagerly; the call graph is constructed on first use, and the disk-backed T. J. Watson Libraries for Analysis (WALA) outcome cache can short-circuit rule checks before the supplier is ever invoked.
   • The results are memoised in an instance-level ConcurrentHashMap keyed by projectPath_packageName_artifact, so they are computed at most once per factory build (a JavaCreator lives for exactly one build — the cache is deliberately not static, so it cannot leak across runs in a long-lived JVM).

2. Preparation — computing allowed packages: Four sources are merged into a single Set<PackagePermission>:

   Source	Rationale
   Essential packages (from YAML)	Framework packages like java, org.aspectj — Ares infrastructure must never be blocked
   Policy-permitted packages	Packages the instructor explicitly allows via regardingPackageImports
   Student's own package	The student must be able to use their own code
   Test-class packages	Infrastructure classes like the test runner must remain accessible

3. Preparation — computing allowed classes: Two sources are merged: essentialClasses ∪ testClasses, each wrapped in a ClassPermission. These classes are exempt from all restrictions.

4. Priority test cases (always-on, generated first):

   • Before the variable cases, four architecture test cases are always generated via JavaArchitectureTestCaseSupported.getPriority(): NATIVE_CODE, AGENT_ATTACH, ENVIRONMENT_ACCESS, and MODULE_SYSTEM. These represent domain-specific checks whose APIs overlap with broader domains (e.g., FILESYSTEM, REFLECTION) — running them first ensures the more specific domain fires before the broader one.

5. Variable test cases (policy-dependent):

   • For each JavaAOPTestCaseSupported value (FILESYSTEM_INTERACTION, NETWORK_CONNECTION, COMMAND_EXECUTION, THREAD_CREATION), creates a JavaAOPTestCase.
   • The matching ResourceAccesses method is selected by an exhaustive switch on the enum constant (e.g., FILESYSTEM_INTERACTION → regardingFileSystemInteractions()). Indexing by ordinal() was deliberately replaced: it would silently pair the wrong supplier with the aspect if the enum were ever reordered, whereas the switch turns a divergence into a compile error.
   • Automatic escalation: If the policy declares no permissions for a category (e.g., no file-system permissions at all), the creator adds a JavaArchitectureTestCase for the same category in addition to the AOP test case. This provides double protection: static analysis catches forbidden application programming interface (API) imports at the bytecode level, and the AOP agent intercepts any calls that slip through. When permissions do exist, only the AOP test is created (static analysis would be too coarse to distinguish allowed from forbidden calls).
   • Phobos test cases are created similarly for FILESYSTEM_INTERACTION, NETWORK_CONNECTION, and TIMEOUT.

6. Fixed test cases (always-on, generated last):

   • Six architecture test cases are always generated regardless of the policy: PACKAGE_IMPORT, TERMINATE_JVM, REFLECTION, SERIALIZATION, CLASS_LOADING, and JNDI_INJECTION. These are retrieved via JavaArchitectureTestCaseSupported.getStatic() (invoked on the arbitrarily chosen constant TERMINATE_JVM, since the method cannot be called on the interface).

---

7 Essential Data — The essentialModel Package

This package manages the lists of packages and classes that Ares itself needs at runtime. These are always permitted, regardless of the security policy — without them, the test framework would block itself.

7.1 EssentialClasses and EssentialPackages Records

Both are isomorphic Java Records with seven sub-lists:

Sub-list	Example content (EssentialClasses.yaml)	Example content (EssentialPackages.yaml)
essentialJava*	(empty)	java
essentialArchunit*	de.tum.cit.ase.ares.api.architecture.java.archunit	(empty)
essentialWala*	de.tum.cit.ase.ares.api.architecture.java.wala	(empty)
essentialAspectJ*	de.tum.cit.ase.ares.api.aop.java.aspectj	org.aspectj
essentialInstrumentation*	de.tum.cit.ase.ares.api.aop.java.instrumentation	de.tum.cit.ase.ares.api.aop.java.aspectj.adviceandpointcut
essentialAres*	18 Ares classes (Creator, Writer, Executer, PolicyReader, etc.)	(empty)
essentialJUnit*	de.tum.cit.ase.ares.api.jupiter	(empty)

Each record provides a getEssentialClasses() / getEssentialPackages() method that concatenates all seven sub-lists into a single flat List<String> using Stream.of(...).flatMap(...) from java.util.stream.

Both records include a Builder with explicit Objects.requireNonNull checks on all seven fields.

7.2 EssentialDataReader Interface

Aspect	Detail
Role	Defines the strategy for reading essential-data YAML files into EssentialClasses and EssentialPackages records.
Pattern	Strategy Pattern. Different implementations can read different file formats.
Key methods	readEssentialClassesFrom(Path) → EssentialClasses and readEssentialPackagesFrom(Path) → EssentialPackages.
Error handling	Provides a default method readerError(identifier, parameter, exception) that returns (rather than throws) a SecurityException with a localised message via Messages.localized(...) — callers throw it explicitly, which keeps the @Nonnull read methods free of an unreachable return null.

7.3 EssentialDataYAMLReader

Aspect	Detail
Implements	EssentialDataReader
Library	Uses Jackson YAML via FileTools.readYamlFile(FileTools.readFile(path), yamlClass).
Error handling	Four specific catch blocks, each producing a localised error message:

Exception	Meaning
StreamReadException	The YAML file has syntax errors
DatabindException	The YAML structure does not match the expected record schema
UnsupportedOperationException	The file format is not supported
IOException	The file was not found or is not readable

---

8 Writing Test Cases — The writer Package

8.1 Writer Interface

Aspect	Detail
Role	Defines the strategy for serialising generated test-case artefacts to disk.
Pattern	Strategy Pattern — one implementation per target language.
Key method	writeTestCases(BuildMode, ArchitectureMode, AOPMode, ..., List<JavaArchitectureTestCase>, List<JavaAOPTestCase>, List<JavaPhobosTestCase>, Path testFolderPath) → List<Path>

8.2 JavaWriter — File Generation

Aspect	Detail
Implements	Writer
Role	Generates four categories of files and writes them into the student project's test folder.

File categories:

Category	Method	What is generated
Architecture files	createJavaArchitectureFiles()	ArchUnit/WALA test source files. Uses the ArchitectureMode to determine which template files to copy and how to format them. Includes both file-system-based (FS) and non-file-system-based (non-FS) files.
AOP files	createJavaAOPFiles()	AspectJ aspects or ByteBuddy Instrumentation configuration files. Uses the AOPMode to select the appropriate templates.
Localisation files	createLocalisationFiles()	Copies messages*.properties files into a resources directory derived from the test folder path: the trailing two segments of testFolderPath are stripped and resources is appended (e.g. src/test/java → src/resources), and the files land under <resources>/ares/api/localization/. This enables localised error messages during test execution.
Phobos files	createPhobosFiles()	Phobos framework configuration files for file-system, network, and timeout test cases.

Template mechanism (Three-Parted Pattern):

All generated files follow the same structure:

Header template  → formatted with (packageName)
Body             → dynamically built from the test-case list
Footer template  → static
→ Concatenated → Written to target path

The FileTools.createThreePartedFormatStringFile(...) utility handles template expansion and file writing. This uniform approach ensures that all generated files are consistent in structure.

---

9 Executing Test Cases — The executer Package

9.1 Executer Interface

Aspect	Detail
Role	Defines the strategy for executing the generated security tests against student code.
Pattern	Strategy Pattern — one implementation per target language.
Key method	executeTestCases(BuildMode, ArchitectureMode, AOPMode, ..., List<JavaArchitectureTestCase>, List<JavaAOPTestCase>)

9.2 JavaExecuter — Runtime Configuration and Execution

Aspect	Detail
Implements	Executer
Role	Configures the runtime agent and then executes all generated test cases.

The execution happens in three ordered steps: agent configuration first, architecture tests second, AOP tests third.

What it does step by step:

1. Configure the AOP agent settings. Seven key-value pairs are written into the agent's configuration via JavaAOPTestCase.setJavaAdviceSettingValue(...):

   Key	Value	Purpose
   buildMode	"MAVEN" or "GRADLE"	Tells the agent which build directory to use
   architectureMode	"ARCHUNIT" or "WALA"	Tells the agent which analysis framework is active
   aopMode	"ASPECTJ" or "INSTRUMENTATION"	Tells the agent which interception mechanism is active
   allowedListedPackages	String[] of essential packages	Packages the agent must never intercept
   allowedListedClasses	String[] of essential + test classes	Classes the agent must never intercept
   restrictedPackage	Student's package name	The package whose code the agent should monitor
   mainClass	Student's main class name	The entry point the agent should focus on

2. Execute architecture tests. Each JavaArchitectureTestCase is executed in order, performing static bytecode analysis. If the student code imports or calls a forbidden API, the test fails immediately.

3. Execute AOP tests. Each JavaAOPTestCase is executed in order, configuring the runtime interception rules. After this step, any forbidden I/O call made by the student code will be intercepted and will throw a SecurityException.

Important: Architecture tests run before AOP tests. This means static violations are detected first (fast, no code execution needed), and runtime enforcement is configured second (covers dynamic behaviour).

---

10 Scanning the Student Project — The projectScanner Package

10.1 ProjectScanner Interface

Defines five scanning methods that auto-detect project metadata:

Method	Returns	What it discovers
scanForBuildMode()	BuildMode	Whether the project uses Maven (pom.xml) or Gradle (build.gradle)
scanForTestClasses()	String[]	Fully qualified names of all classes in the test source directory containing @Test or @Property annotations, or extending JUnit 3's TestCase
scanForPackageName()	String	The most frequently used non-reserved package: taken from the production sources, otherwise from the compiled production output, otherwise the configured default
scanForMainClassInPackage()	String	The class containing public static void main(String[])
scanForTestPath()	Path	The file system path to the test source directory

10.2 JavaProjectScanner

Aspect	Detail
Implements	ProjectScanner
Technique	JavaParser-backed source analysis. Walks the .java files under the discovered source roots and reads the parsed syntax tree; the compiled output is read with ArchUnit's ClassFileImporter where the sources yield nothing.

What is read from the syntax tree:

Fact	Read from	Used by
Package declaration	the compilation unit's PackageDeclaration	scanForPackageName()
Type declarations	the top-level TypeDeclarations, nested types included	scanForMainClassInPackage(), scanForTestClasses()
main method	a public static void main(String[]) declaration, varargs included	scanForMainClassInPackage()
Test classes	a @Test or @Property annotation, or a JUnit 3 TestCase supertype resolved through the imports of the file	scanForTestClasses()

Resolving the supertype through the imports is why this is not a regex: extends TestCase names a type, and which type it names depends on what the file imported.

Scanning pipeline:

ProjectSourcesFinder.discover(projectRoot, mode)      → BuildToolConfiguration
  → configuration.productionSourceRoots() / testSourceRoots()
    → Files.walk(root), filtered to *.java and sorted
      → JavaParser.parse(file) → CompilationUnit
  and, where the sources answer nothing:
  → ClassFileImporter().importPath(productionOutputRoot)

The sort is not cosmetic: it is what makes two runs over one project agree. The legacy findProjectSourcesPath() route still exists for callers that predate BuildToolConfiguration, and differs in kind: it returns the descriptor's own string, relative and unvalidated, where discover(...) canonicalises every root and refuses one that escapes the project.

scanForPackageName() algorithm: Resolution runs in three steps, each reached only when the previous one finds nothing at all.

1. Production sources. Reserved infrastructure prefixes are filtered out first (via ReservedPackageGuard.reservedPrefixOf(...)), so a package inside a trusted namespace cannot become the derived enforcement scope. The frequency of every remaining package declaration is counted and the most common one wins. This heuristic works because in a typical student project the main source package appears in the majority of files.
2. Compiled production output. Only top-level classes are counted, so a package is not weighted by how many nested or anonymous classes it happens to contain; nesting is read from the class file rather than from the $ in the binary name, which is a legal identifier character. This step covers a project whose build descriptor the source-root discovery cannot parse, because the build tool writes its output to the conventional directory the scanner reads.
3. The configured default (see Section 10.3), with a warning naming the roots that were searched.

Step 1 is skipped entirely when the discovered source roots are not known to be the whole of the main source set. Two things lead there: a descriptor can declare a root this reader cannot resolve, such as a computed list, and it can declare one that resolves cleanly but names a directory the project does not contain, which is passed over. Either way BuildToolConfiguration.productionRootsComplete() reports it, for Maven as well as for Gradle. Counting declarations across part of a project produces an answer indistinguishable from one taken across all of it, so a partial set is not counted at all and the compiled output is read instead.

The derived package is a heuristic. What turns it into a boundary is the check that follows it.

Before enforcement is armed, requireDerivedScopeToCoverTheProject() reads the compiled production output and refuses the run unless every executable top-level class declares a non-blank, non-reserved package that is the derived scope or lies below it, compared on segment boundaries so that de.tum.cit.aet does not swallow de.tum.cit.aetevil. A class the scope leaves out, a class in the default package, a class in a reserved package, and an output root that exists but cannot be read are each refused by name. This runs on the policy-free path only: a pinned policy may deliberately supervise part of the output, and narrowing it is then the instructor's decision.

An output root holding nothing passes, with a warning, and so does one holding only package-info or module-info. There is then no supervisable class, so enforcement is vacuous rather than mis-scoped, and an exercise whose supervised package is still empty must not fail for being empty. The generated test does the same: JavaArchunitSupervisedClasses warns and analyses an empty set rather than refusing. In both places that log line is the only signal, and a suite that analyses nothing reports success.

That closes the case where a decoy package is voted the scope while the assignment runs beside it. Three things it still does not establish.

The vote is influenceable by whoever can add files to the project, and in an Artemis exercise that includes the student. The check above refuses a scope that leaves compiled classes out, but not one drawn around them: a scope that covers everything passes by construction. The package-import allow-list no longer follows the scope for that reason, and names the packages the validated output declares instead.

The output directory is assumed, not read. Step 2 and the check both look in target/classes or build/classes/java/main, so a build that writes its output elsewhere is not followed there. Together with the vacuous pass above, that is the sharp edge of this section: a project whose output goes somewhere else looks exactly like a project that compiled nothing, and both pass with the same warning. An exercise configured with a custom output destination is therefore not enforced by the derived path at all, and nothing fails to say so. Such an exercise must declare its scope in a policy.

The last-resort default guarantees nothing by itself. If the project does not contain it, the analysis path resolves to a directory that does not exist. Where anything at all is compiled, the check above catches it: those classes lie outside the default, so the run is refused by name. Where nothing is compiled it does not, and the warning is again all a reader gets. During generation, before anything is compiled, the same is true.

An exercise that needs a scope it can rely on declares its package in the security policy. The scanner is then not consulted at all, which is the only version of this that cannot be steered from the submission.

scanForTestClasses() algorithm: Scans only the test source directory (see scanForTestPath()) and returns every class whose file contains a @Test / @Property annotation or extends TestCase.

scanForMainClassInPackage() algorithm: Collects all classes with a main method → prefers a class named Main or Application → otherwise returns the first match → defaults to "Main".

scanForTestPath() algorithm: Answers the first discovered test source root; without a build configuration it accepts the conventional src/test/java, or a bare test/ directory for the Artemis Gradle layout, and otherwise falls back to the literal src/test/java whether or not it exists. That fall-back is a placeholder forced by the non-null return type rather than a claim, and no production code currently consults this method.

10.3 JavaProgrammingExerciseProjectScanner

Aspect	Detail
Extends	JavaProjectScanner
Purpose	Overrides defaults for TUM Artemis programming exercises.

Override	Default in JavaProjectScanner	Override in JavaProgrammingExerciseProjectScanner
Default package	"" (empty string)	"de.tum.cit.aet"
Default main class	"Main"	"Main" (unchanged)

When the base scanner finds no package or main class, these TUM-specific defaults ensure reasonable behaviour for Artemis-hosted exercises.

---

11 ArchUnit Integration — The specific Package

This package contains a single class: PathLocationProvider.

Aspect	Detail
Implements	com.tngtech.archunit.junit.LocationProvider
Purpose	Tells ArchUnit where to find the compiled student .class files. Normally ArchUnit analyses its own classpath, but student submissions may be compiled to a custom directory (e.g., by a Learning Management System).
Mechanism	Reads the @StudentCompiledClassesPath annotation from the test class → converts its value() to a Path → wraps it in an ArchUnit Location.
Error handling	Throws a SecurityException if the test class is not annotated with @StudentCompiledClassesPath.

---

12 The StudentCompiledClassesPath Annotation

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.TYPE)
public @interface StudentCompiledClassesPath {
    @Nonnull String value();  // file system path to compiled classes
}

Aspect	Detail
Target	Test classes that use PathLocationProvider
Retention	RUNTIME — the annotation is available to ArchUnit's reflection-based LocationProvider at test time
Usage	@StudentCompiledClassesPath("build/classes/java/main") on the instructor's test class

---

13 Processing Pipeline (Overview)

The following diagram shows the end-to-end flow from a SecurityPolicy object to enforced security tests:

                    SecurityPolicy (parsed record)
                           │
                           ▼
           ┌───────────────────────────────┐
           │ TestCaseAbstractFactory       │  ← Constructor (Template Method)
           │ AndBuilder                    │
           ├───────────────────────────────┤
           │ Step 1: Inject tools          │  Creator, Writer, Executer,
           │                               │  EssentialDataReader, ProjectScanner
           │ Step 2: Resolve modes         │  BuildMode, ArchitectureMode, AOPMode
           │ Step 3: Load essential data   │  ← EssentialDataYAMLReader reads YAML
           │ Step 4: Extract config        │  ← from Policy (fail-closed) or Scanner
           │ Step 5: Guard package         │  ← ReservedPackageGuard.validatePackage()
           │ Step 6: Create test cases     │  ← JavaCreator generates tests
           └───────────┬───────────────────┘
                       │
        ┌──────────────┼──────────────┐
        ▼              ▼              ▼
   Architecture    AOP Test      Phobos Test
   TestCases       Cases         Cases
        │              │              │
        └──────────────┼──────────────┘
                       │
          ┌────────────┴────────────┐
          ▼                         ▼
   writeTestCases()          executeTestCases()
          │                         │
          ▼                         ▼
   JavaWriter                 JavaExecuter
   ├─ Architecture files      ├─ Configure agent (7 settings)
   ├─ AOP files               ├─ Run architecture tests (static)
   ├─ Localisation files      └─ Run AOP tests (dynamic)
   └─ Phobos files
          │                         │
          ▼                         ▼
   List<Path>               SecurityException if
   (generated files)         student code violates policy

---

14 End-to-End Example

1. Instructor writes SecurityConfiguration.yaml:

thisPolicyFileCompliesToThePolicyVersion: 1
regardingTheSupervisedCode:
  theFollowingProgrammingLanguageConfigurationIsUsed: JAVA_USING_GRADLE_WALA_AND_INSTRUMENTATION
  theSupervisedCodeUsesTheFollowingPackage: com.student
  theMainClassInsideThisPackageIs: "Main"
  theFollowingClassesAreTestClasses:
    - com.student.test.SecurityTest
  theFollowingResourceAccessesArePermitted:
    regardingFileSystemInteractions: []
    regardingNetworkConnections: []
    regardingCommandExecutions: []
    regardingThreadCreations: []
    regardingPackageImports: []
    regardingTimeouts:
      - timeout: 5000

2. Instructor writes a JUnit test:

import de.tum.cit.ase.ares.api.Policy;
import org.junit.jupiter.api.Test;

class SecurityTest {
    @Test
    @Policy(value = "SecurityConfiguration.yaml",
            withinPath = "classes/java/main/com/student")
    void studentCodeMustNotAccessFileSystem() {
        com.student.Main.main(new String[]{});
    }
}

3. What happens inside this package at runtime:

1. The SecurityPolicyJavaDirector calls JavaTestCaseFactoryAndBuilder.builder() with all 12 parameters.
2. The TestCaseAbstractFactoryAndBuilder constructor begins:
   • Mode Resolution: BuildMode.GRADLE, ArchitectureMode.WALA, AOPMode.INSTRUMENTATION.
   • Essential Data: EssentialDataYAMLReader reads EssentialPackages.yaml (→ java, org.aspectj, etc.) and EssentialClasses.yaml (→ 18 Ares classes).
   • Policy Extraction: packageName = "com.student", mainClass = "Main", testClasses = ["com.student.test.SecurityTest"], resourceAccesses = all empty (fully restrictive).
3. JavaCreator.createTestCases() runs:
   • Computes the classpath: build/classes/java/main.
   • Imports JavaClasses and builds a WALA CallGraph (cached).
   • Computes allowed packages: {java, org.aspectj, com.student, com.student.test}.
   • Priority test cases: NATIVE_CODE, AGENT_ATTACH, ENVIRONMENT_ACCESS, and MODULE_SYSTEM architecture tests (always-on, generated first).
   • Variable test cases: Since all resource-access lists are empty, for each category (FS, Network, Command, Thread) both an ArchitectureTestCase and an AOPTestCase are created (automatic escalation).
   • Fixed test cases: PACKAGE_IMPORT, TERMINATE_JVM, REFLECTION, SERIALIZATION, CLASS_LOADING, and JNDI_INJECTION architecture tests (always-on).
4. JavaWriter.writeTestCases() serialises ArchUnit test files, Instrumentation configuration, and Phobos configs into src/test/java/com/student/, and localisation properties into src/resources/ares/api/localization/.
5. JavaExecuter.executeTestCases():
   • Configures the ByteBuddy agent with restrictedPackage = "com.student" and allowedListedClasses.
   • Runs the WALA-based architecture tests → if com.student.Main calls java.io.File anywhere in the call graph, the test fails.
   • Configures the AOP interception rules → when Main.main() runs, any File.read() call is intercepted and throws SecurityException.

---

15 Troubleshooting

Problem	Possible Cause	Solution
NullPointerException in JavaTestCaseFactoryAndBuilder.build()	A mandatory builder parameter (creator, writer, executer, essentialDataReader, projectScanner, or an essential-data path) was not set	Ensure all 7 mandatory parameters are set before calling build()
NullPointerException with message "essentialClassesPath must not be null"	The path to EssentialClasses.yaml was not provided	Verify that the director or builder sets essentialClassesPath and essentialPackagesPath
SecurityException from EssentialDataYAMLReader — "read failed" or "data bind failed"	The EssentialClasses.yaml or EssentialPackages.yaml file is malformed or missing	Check that the YAML files exist at the expected classpath location and have the correct schema (7 list fields each)
Architecture tests pass but runtime enforcement is missing	The Java agent JAR is not loaded via -javaagent	See the Maven or Gradle walkthrough for agent setup
Scanner detects the wrong package name	The most-frequent-package heuristic picks a utility package instead of the student's main package	Specify theSupervisedCodeUsesTheFollowingPackage explicitly in the security policy YAML
Scanner finds no test classes	Java source files do not contain @Test or @Property annotations (or extends TestCase), or files are not under the test source directory	Specify theFollowingClassesAreTestClasses explicitly in the security policy YAML (note: with a policy present, test classes come only from the policy — the scanner is not consulted)
SecurityException from PathLocationProvider — "can only be used on classes annotated with…"	The test class using PathLocationProvider is missing the @StudentCompiledClassesPath annotation	Add @StudentCompiledClassesPath("build/classes/java/main") to the test class
Call-graph analysis is slow	WALA call-graph construction is computationally expensive for large projects	Switch to ArchitectureMode.ARCHUNIT (rule-based, faster but less precise), or ensure the cache is not invalidated between runs

---

16 Glossary

Term	Meaning
Abstract Factory	A design pattern that provides an interface for creating families of related objects (here: architecture tests and AOP tests) without specifying their concrete classes.
Template Method	A design pattern where an abstract class defines the skeleton of an algorithm in a method, deferring some steps to subclasses. Here, the constructor of TestCaseAbstractFactoryAndBuilder fixes the 6-step initialisation sequence.
Strategy Pattern	A design pattern that defines a family of interchangeable algorithms. Here: Creator, Writer, Executer, ProjectScanner, and EssentialDataReader are all strategy interfaces with swappable implementations.
AOP (Aspect-Oriented Programming)	A technique for intercepting method calls at defined points ("pointcuts") without modifying the original code. Ares uses AOP to block forbidden I/O operations at runtime.
ArchUnit	A Java library for checking architecture rules on compiled bytecode (e.g., "no class in package X may call class Y").
WALA	A static analysis framework that builds inter-procedural call graphs to detect forbidden API usage even through chains of method calls.
AspectJ	A compile-time AOP framework that weaves interception code directly into bytecode.
Instrumentation (Java Agent)	A runtime AOP approach using the java.lang.instrument API and ByteBuddy. A Java agent modifies class bytecode at load time.
ByteBuddy	A library for creating and modifying Java classes at runtime, used by Ares to implement the instrumentation agent.
Phobos	A further test framework within Ares that generates test cases for file-system interactions, network connections, and timeout enforcement.
Essential Packages / Classes	Packages and classes that Ares itself needs at runtime (e.g., JUnit, ArchUnit, ByteBuddy, Ares internals). These are always permitted, regardless of the security policy, to prevent the test framework from blocking itself.
Automatic Escalation	When the security policy declares no permissions for a resource category (e.g., no file-system permissions), the JavaCreator generates both an architecture test (static) and an AOP test (dynamic) for that category, providing double protection.
Three-Parted File	A generated file consisting of a Header template, a dynamically built Body, and a Footer template. Used by the Writer to produce all test-case source files.
ResourceAccesses	A record from the policy package that holds six lists of permissions: file-system, network, command, thread, package-import, and timeout. The factory uses this record to decide which test cases to generate.