Cutting context window pollution with silent test runners

October 22, 2026 :: Wietse Venema

Most AI coding agents send terminal output straight to the LLM context window. While human developers enjoy watching green checkmarks scroll past, an agent consumes every passing assertion as input tokens, which degrades performance and increases costs. To keep the context window clean without losing failure details, I wrap my test commands so they print a single confirmation line on success and show the full output only when a test fails.

Writing the test runner macro

I implemented this with a reusable macro in my project's Makefile:

# Macro to run test commands quietly.
# Suppresses stdout and stderr on success to save LLM context window tokens.
# On failure, prints the captured output and exits with the original exit code.
define RUN_TEST
	@TMPFILE=$$(mktemp); \
	if $(1) > "$$TMPFILE" 2>&1; then \
		echo "✓ $(2) passed. Output suppressed to save context window space (failures print full output). Prefer Make targets over calling test runners directly."; \
		rm -f "$$TMPFILE"; \
	else \
		EXIT_CODE=$$?; \
		cat "$$TMPFILE"; \
		rm -f "$$TMPFILE"; \
		exit $$EXIT_CODE; \
	fi
endef

Here's how the shell commands in the macro work:

  • TMPFILE=$$(mktemp) creates a temporary file in /tmp to buffer the output.
  • $(1) > "$$TMPFILE" 2>&1 runs the test command passed as the first argument ($(1)) and redirects both standard output (file descriptor 1) and standard error (file descriptor 2) into the temporary file.
  • if ... then checks the exit code of the test command.
  • If the command exits with 0, the shell prints the confirmation line using the suite label passed as $(2) and runs rm -f "$$TMPFILE" (using -f to force removal without prompting) to delete the temporary file.
  • If the command exits with a non-zero status, EXIT_CODE=$$? saves the exit code, cat "$$TMPFILE" prints the captured output, rm -f "$$TMPFILE" removes the temporary file, and exit $$EXIT_CODE exits Make with the original failure code.

The echo message is written directly for the agent. If a test target prints nothing at all (or only prints ✓ passed), coding agents sometimes assume the tests didn't actually run and bypass make to run pytest or go test directly. Telling the agent in the output itself why the logs were suppressed, and reminding it to use Make targets, stops it from working around the wrapper.

It also helps to add the same rule to your project's agent instructions (such as GEMINI.md or AGENTS.md): 

When running tests, prefer Make targets (such as `make test`) over calling test runners directly.

Using the macro in your Makefile

Here's how I wrap Go and Python test commands with RUN_TEST:

.PHONY: test test-unit test-python

# Run all test suites quietly
test:
	$(call RUN_TEST,go test -race -p 1 ./...,Go tests)
	$(call RUN_TEST,uv run pytest,Python tests)

# Unit test suite
test-unit:
	$(call RUN_TEST,go test -race ./pkg/...,Go unit tests)

# Python test suite
test-python:
	$(call RUN_TEST,uv run pytest,Python tests)

The targets omit -v so a failure only outputs the failing test diagnostics rather than every passing test in the suite. For Go tests, -race enables the race detector, and -p 1 runs package tests sequentially so failure logs don't interleave.

Verifying the output

Run the test suite with make:

make test

When all tests pass, make test prints two confirmation lines:

✓ Go tests passed. Output suppressed to save context window space (failures print full output). Prefer Make targets over calling test runners directly.
✓ Python tests passed. Output suppressed to save context window space (failures print full output). Prefer Make targets over calling test runners directly.

And if a test fails, you get the full failure output and exit code:

--- FAIL: TestFlightComputer (0.01s)
    flight_test.go:42: expected altitude 10000, got 9850
FAIL
FAIL	example.com/flight	0.015s
make: *** [test] Error 1

If you want to try this out in your project, you can paste this URL into your agent directly.

Signature