Introduction

File and directory discovery (T1083) is a core reconnaissance technique. This post documents practical methods for enumerating filesystems, from passive information gathering to active command execution.

Method 1: Passive Information Gathering

1.1 Web Crawling and Sitemap Analysis

Objective: Identify publicly accessible files and directory structure

Tools:

• curl / wget — Fetch pages and analyze links
• robots.txt — Check for directory hints
• sitemap.xml — Public directory mapping

Example:

# Fetch robots.txt for directory hints
curl -s https://target.com/robots.txt

# Parse sitemap for accessible paths
curl -s https://target.com/sitemap.xml | grep -oP '<loc>\K[^<]*'

Output: /admin/, /api/, /backup/, /uploads/

Purpose: Tactical reference guide for querying OpenSearch audit logs to identify anomalies, security events, and suspicious activity.

Core Concept

OpenSearch audit logs record all REST API requests (authentication, data access, privilege changes). Systematic querying of these logs reveals:

• Unauthorized access attempts
• Unusual data access patterns
• Privilege escalation
• Service account abuse
• Automated scanning activity

Index Structure & Naming

OpenSearch stores audit logs in daily indices:

security-auditlog-YYYY.MM.DD

Examples:

security-auditlog-2026.03.04  (today)
security-auditlog-2026.03.03  (yesterday)
security-auditlog-2026.03-*   (all March 2026)
security-auditlog-*           (all indices)

Use wildcard patterns to query multiple days at once.