LinuxCBT SHELL Edition

Course Objective

Shell Scripting

Anonymize Logs
- Locate target logs for processing
- Identify sensitive data
- Write script to anonymize sensitive data
- Work with Logs and Scripts in anonymized mode

Process N Logs
- Discuss limitiations of default invocation
- Force N number of CLI arguments
- Confirm CLI arguments
- Ensure existence of supplied input file(s)
- Test input logic

Storage Check
- Extend script to handle new functions
- Calculate available storage
- Ensure storage can support generated output
- Test supplied input against available storage
- Debug as needed

Fixed-Length Records
- Create script to check input field lengths
- Use Regular Expressions to check input
- Except non-standard records
- Process records
- Identify mismatches
- Integrate fixed-length checks into main script
- Test and debug where applicable

Multiple Spaces | Tab Delimiter
- Retrofit script to process variable whitespace input
- Use SED to parse input stream
- Test variable input streams against Awk && SED
- Debug as needed

Apache Timestamp Conversion
- Examine default log entries
- Compare with ISO standard and MySQL support
- SHELL-test timestamp conversions using: Awk
- Alter script to transform timestamp to numeric ISO standard
- Rearrange time values to match ISO standard
- Confirm conversions

Apache Log Stats
- Use Apache Logs as Database
- Ascertain: Total Bytes
- Summarize Largest Requests
- Average Bytes across Requests
- Top 5 Largest Requests
- Most Frequent Status Codes
- Top IPs

CSV Output | MySQL Import
- Retrofit anonymize script to yield CSV
- Enclose all fields with double quotes
- Generate output (CSV) and confirm
- Import CSV data into MySQL
- Test efficacy

Free Memory & Swap
- Define reusable stub
- Integrate with existing processes
- Vary thresholds
- Execute and evalute

Get Data from Instances
- Write script to ascertain remote node's network details
- Create stub script to parallelize calls via: parallel-ssh
- Test parallelization on targets
- Log data
- Cron script
- Extend script to yield: Memory && storage details
- Confirm results

Find Files of Interest
- Stub 'find' to locate files of interest
- Vary input and searches
- Evaluate

Identify Common Bottlenecks
- Identify helper utilities
- Parse output to expose interesting metrics
- Summarize metrics

TCP States | Apache Stats
- Grab TCP states from targets
- Summarize output
- Ascertain Apache Stats across instances
- Evaluate results