Redelk - Tardily Deployable Tool For Cherry-Red Teams Used For Tracking Too Alarming Nigh Blueish Squad Activities Equally Good Equally Improve Usability Inwards Long Term Operations
Red Team's SIEM - slow deployable tool for Red Teams used for tracking together with alarming nearly Blue Team activities every bit good every bit improve usability for the Red Team inward long term operations.
Initial world free at BruCON 2018:
- Video: https://www.youtube.com/watch?v=OjtftdPts4g
- Presentation slides: https://github.com/outflanknl/Presentations/blob/master/MirrorOnTheWall_BruCon2018_UsingBlueTeamTechniquesinRedTeamOps_Bergman-Smeets_FINAL.pdf
Goal of the project
Short: a Red Team's SIEM.
Longer: a Red Team's SIEM that serves 3 goals:
- Enhanced usability together with overview for the cerise squad operators past times creating a primal place where all relevant operational logs from multiple teamservers are collected together with enriched. This is peachy for historic searching inside the performance every bit good every bit giving a read-only persuasion on the performance (e.g. for the White Team). Especially useful for multi-scenario, multi-teamserver, multi-member together with multi-month operations. Also, super slow ways for viewing all screenshots, IOCs, keystrokes output, etc. \o/
- Spot the Blue Team past times having a primal place where all traffic logs from redirectors are collected together with enriched. Using specific queries its similar a shot possible to unwrap that the Blue Team is investigating your infrastructure.
- Out-of-the-box usable past times existence slow to install together with deploy, every bit good every bit having ready made views, dashboards together with alarms.
RedELK uses the typical components Filebeat (shipping), Logstash (filtering), Elasticsearch (storage) together with Kibana (viewing). Rsync is used for a instant syncing of teamserver data: logs, keystrokes, screenshots, etc. Nginx is used for authentication to Kibana, every bit good every bit serving the screenshots, beaconlogs, keystrokes inward an slow way inward the operator's browser.
H5N1 laid of python scripts are used for heavy enriching of the log data, together with for for Blue Team detection.
Supported tech together with requirements
RedELK currently supports:
- Cobalt Strike teamservers
- HAProxy for HTTP redirector data. Apache back upwards is expected soon.
- Tested on Ubuntu xvi LTS
In the 'general' section:
log-format frontend:%f/%H/%fi:%fp\ backend:%b\ client:%ci:%cp\ GMT:%T\ useragent:%[capture.req.hdr(1)]\ body:%[capture.req.hdr(0)]\ request:%rdeclare capture asking len 40000 http-request capture req.body id 0 capture asking header User-Agent len 512Installation
First fourth dimension installation
Adjust
./certs/config.cnf to include the correct details for the TLS certificates. Once done, run: initial-setup.sh This volition practise a CA, generate necessary certificates for secure communication betwixt redirs, teamserver together with elkserver together with generates a SSH keypair for secure rsync authentication of the elkserver to the teamserver. It also generates teamservers.tgz, redirs.tgz together with elkserver.tgz that comprise the installation packages for each component. Rerunning this initial setup is non required. But if you lot desire novel certificates for a novel operation, you lot tin give the axe only run this again.Installation of redirectors
Copy together with extract redirs.tgz on your redirector every bit portion of your cerise squad infra deployment procedures. Run:
install-redir.sh $FilebeatID $ScenarioName $IP/DNS:PORT- $FilebeatID is the identifier of this redirector inside filebeat.
- $ScenarioName is the cite of the laid on scenario this redirector is used for.
- $IP/DNS:PORT is the IP or DNS cite together with port where filebeat logs are shipped to.
Installation of teamserver
Copy together with extract teamservers.tgz on your Cobalt Strike teamserver every bit portion of your cerise squad infra deployment procedures. Run:
install-teamserver.sh $FilebeatID $ScenarioName $IP/DNS:PORT- $FilebeatID is the identifier of this teamserver inside filebeat.
- $ScenarioName is the cite of the laid on scenario this teamserver is used for.
- $IP/DNS:PORT is the IP or DNS cite together with port where filebeat logs are shipped to.
Installation of ELK server
Copy together with extract elkserver.tgz on your RedELK server every bit portion of your cerise squad infra deployment procedures. Run:
install-teamserver.sh This script volition laid the timezone (default Europe/Amsterdam), install logstash, elasticsearch, kibana together with dependencies, install required certificates, deploy the logstash configuration together with required custom ruby enrichment scripts, download GeoIP databases, install Nginx, configure Nginx, practise a local user 'redelk' amongst the before generated SSH keys, install the script for rsyncing of remote logs on teamservers, install the script used for creating of thumbnails of screenshots, install the RedELK configuration files, install crontab file for RedELK tasks, install GeoIP elasticsearch plugins together with conform the template, install the python enrichment scripts, together with finally install the python blueish squad detection scripts.You are non done yet. You demand to manually larn inward the details of your teamservers inward
/etc/cron.d/redelk, every bit good every bit melody the config files inward /etc/redelk (see department below).Setting upwards enrichment together with detection
On the ELK server inward the
/etc/redelk directory you lot tin give the axe unwrap several files that you lot tin give the axe usage to melody your RedELK instance for improve enrichments together with improve alarms. These files are:/etc/redelk/iplist_customer.conf: world IP addresses of your target, 1 per line. Including an address hither volition laid a tag for applicable records inward the redirhaproxy-* index./etc/redelk/iplist_redteam.conf: world IP addresses of your cerise team, 1 per line. Convenient for identifying testing done past times cerise squad members. Including an address hither volition laid a tag for applicable records inward the redirhaproxy-* index./etc/redelk/iplist_unknown.conf: world IP addresses of gateways that you lot are non for certain nearly yet, but don't desire to live warned nearly again. One per line. Including an address hither volition laid a tag for applicable records inward the redirhaproxy-* index./etc/redelk/known_sandboxes.conf: beacon characteristics of known AV sandbox systems. One per line. Including information hither here volition laid a tag for applicable records inward the rtops-* index./etc/redelk/known_testsystems.conf: beacon characteristics of known seek systems. One per line. Including information hither here volition laid a tag for applicable records inward the rtops-* index./etc/redelk/alarm.json.config: details required for alarms to work. This includes API keys for online services (Virus Total, IBM X-Force, etc) every bit good every bit the SMTP details required for sending alarms via e-mail.
./RedELK/elkserver/etc/redelk.To alter the authentication onto Nginx, alter
/etc/nginx/htpasswd.users to include your preferred credentials. Or ./RedELK/elkserver/etc/nginx/htpasswd.users prior to initial setup.Under the hood
If you lot desire to receive got a aspect nether the hood on the ELK server, receive got a aspect at the redelk cron file inward
/etc/cron.d/redelk. It starts several scripts inward /usr/share/redelk/bin/. Some scripts are for enrichment, others are for alarming. The configuration of these scripts is done amongst the config files inward /etc/redelk/. There is also heavy enrichment done (including the generation of hyperlinks for screenshots, etc) inward logstash. You tin give the axe banking firm gibe that out direct cast the logstash config files inward /etc/logstash/conf.d/.Current Blue Planet together with features on todo-list
This projection is withal inward alpha phase. This agency that it plant on our machines together with our environment, but no extended testing is performed on unlike setups. This also agency that naming together with construction of the code is withal dependent champaign to change.
We are working (and you lot are invited to contribute) on the next features for side past times side versions:
- Include the existent external IP address of a beacon. As Cobalt Strike has no noesis of the existent external IP address of a beacon session, nosotros demand to larn this cast the traffic index. So far, nosotros receive got non constitute a truthful 100% reliable way for doing this.
- Support for Apache redirectors. Fully tested together with working filebeat together with logstash configuration files that back upwards Apache based redirectors. Possibly additional custom log configuration needed for Apache. Low priority.
- Solve rsyslog max log job issue. Rsyslog (default syslog service on Ubuntu) breaks long syslog lines. Depending on the CS profile you lot use, this tin give the axe drib dead an issue. As a result, the parsing of unopen to of the fields are properly parsed past times logstash, together with therefore non properly included inward elasticsearch.
- Ingest manual IOC data. When you lot are uploading a document, or something else, exterior of Cobalt Strike, it volition non live included inward the IOC list. We desire an slow way to receive got these manual IOCs also included. One way would live to larn inward the information manually inward the activity log of Cobalt Strike together with receive got a logstash filter to scrape the information from there.
- Ingest e-mails. Create input together with filter rules for IMAP mailboxes. This way, nosotros tin give the axe usage the same slow ELK interface for having an overview of sent emails, together with replies.
- User-agent checks. Tagging together with alarming on suspicious user-agents. This volition belike live divided inward hardcoded materials similar curl, wget, etc connecting amongst the proper C2 URL's, but also to a greater extent than dynamic analysis of suspicious user-agents.
- DNS traffic analyses. Ingest, filter together with inquiry for suspicious activities on the DNS level. This volition receive got considerable piece of work due to the large sum of noise/bogus DNS queries performed past times scanners together with online DNS inventory services.
- Other alert channels. Think Slack, Telegram, whatever other way you lot desire for receiving alarms.
- Fine grained authorisation. Possibility for blocking for certain views, searches, together with dashboards, or masking for certain details inward unopen to views. Useful for situations where you lot don't desire to give out all information to all visitors.
Usage
First fourth dimension login
Browse to your RedELK server's IP address together with login amongst the credentials from Nginx (default is redelk:redelk). You are similar a shot inward a Kibana interface. You may live asked to practise a default index for kibana. You tin give the axe direct whatever of the available indices, it doesn't thing which 1 you lot pick.
There are belike 2 things you lot desire to practise here: aspect at dashboards, or aspect together with search the information inward to a greater extent than detail. You tin give the axe switch betwixt those views using the buttons on the left bar (default Kibana functionality).
Dashboards
Click on the dashboard icon on the left, together with you'll live given 2 choices: Traffic together with Beacon.
Looking together with searching information inward detail
Click on the Discover push to aspect at together with search the information inward to a greater extent than detail. Once there, click the fourth dimension arrive at you lot desire to usage together with click on the 'Open' push to usage 1 of the prepared searches amongst views.
Beacon data
When selecting the search 'TimelineOverview' you lot are presented amongst an slow to usage persuasion on the information from the Cobalt Strike teamservers, a fourth dimension job of beacon events if you lot like. The persuasion includes the relevant columns you lot desire to have, such every bit timestamp, testscenario name, username, beacon ID, hostname, OS together with OS version. Finally, the total message from Cobalt Strike is shown.
You tin give the axe modify this search to your liking. Also, because its elasticsearch, you lot tin give the axe search all the information inward this index using the search bar.
Clicking on the details of a tape volition exhibit you lot the total details. An of import champaign for usability is the beaconlogfile field. This champaign is an hyperlink, linking to the total beacon log file this tape is from. Its allows you lot to aspect at the beacon transcript inward a bigger windows together with usage CTRL+F inside it.
Screenshots
RedELK comes amongst an slow way of looking at all the screenshots that were made from your targets. Select the 'Screenshots' search to larn this overview. We added 2 large usability things: thumbnails together with hyperlinks to the total pictures. The thumbnails are at that spot to chop-chop scroll through together with give you lot an immediate impression: oftentimes you lot withal retrieve what the screenshot looked like.
Keystrokes
Just every bit amongst screenshots, its real handy to receive got an slow overview of all keystrokes. This search gives you lot the kickoff lines of cententi, every bit good every bit in 1 trial to a greater extent than an hyperlink to the total keystrokes log file.
To larn a quick listing of all IOCs, RedELK comes amongst an slow overview. Just usage the 'IOCs' search to larn this list. This volition acquaint all IOC information from Cobalt Strike, both from files together with from services.
You tin give the axe chop-chop export this listing past times hitting the 'Reporting' push inward the top bar to generate a CSV of this exact view.
Logging of RedELK
During installation all actions are logged inward a log file inward the electrical current working directory.
During operations, all RedELK specific logs are logged on the ELK server inward
/var/log/redelk. You belike solely demand this for troubleshooting.Authors together with contribution
This projection is developed together with maintained by:
- Marc Smeets (@smeetsie on Github together with @mramsmeets on Twitter).
- Mark Bergman (@xychix on Github together with Twitter)
