I passed a compliance audit. Then the auditor came back with one question: “How were these reports generated?” I had the reports. I had the scripts that generated them. But the scripts had no logging — no timestamps, no record of what ran, no proof that the output matched the input.
The Setup
For a SOC 2-style audit, I needed to submit evidence: branch protection reports, team access lists, user permissions, database access controls, and commit logs. I had scripts that pulled this data from GitHub, a database admin tool, and Google Workspace APIs.
The scripts worked. They generated CSVs and uploaded them to a shared drive. Done, right?
The Auditor’s Question
The auditor didn’t question the data. They questioned the process:
- When exactly was this report generated?
- What API calls produced this data?
- How do I know the script didn’t filter or modify the results?
- Can you reproduce this output?
The scripts printed nothing. No start time, no API URLs, no record counts, no checksums. They ran silently and produced files.
What I Fixed
Every script got verbose, timestamped logging:
echo "[$(date -u +%Y-%m-%dT%H:%M:%SZ)] Fetching team members from GitHub API..."
echo "[$(date -u +%Y-%m-%dT%H:%M:%SZ)] GET /orgs/{org}/teams?per_page=100 - page 1"
echo "[$(date -u +%Y-%m-%dT%H:%M:%SZ)] Retrieved 47 teams"
echo "[$(date -u +%Y-%m-%dT%H:%M:%SZ)] GET /orgs/{org}/teams/{team}/members - page 1"
echo "[$(date -u +%Y-%m-%dT%H:%M:%SZ)] Total members across all teams: 156"
echo "[$(date -u +%Y-%m-%dT%H:%M:%SZ)] Writing to team-access-report.csv (156 rows)"
The log output became part of the evidence package alongside the reports.
The Bugs I Found Along the Way
Adding logging exposed real problems:
Missing pagination. All four GitHub API calls only fetched the first page (30 results). With per_page=100 and proper pagination, the team list went from 30 to 47. I’d been submitting incomplete data.
Wrong API endpoint. The database admin tool had changed its API in v3.x — the workspace endpoint moved. The script was hitting the old path and getting empty results. The “access report” showed zero database users, which the auditor hadn’t questioned yet — but would have.
Accidental uploads. Running the script locally uploaded files to the service account’s personal Google Drive instead of the shared audit folder. I added --local-only as the default, with --force-upload required to actually push to the drive.
Takeaway
Compliance scripts aren’t just data extractors — they’re evidence chains. If a script produces a report that an auditor will review, the script’s execution log is as important as the report itself. Add timestamps, API URLs, record counts, and checksums. The auditor won’t trust a CSV that appeared from nowhere.