Blue Teaming at Pros Vs Joes CTF, BSidesLV 2016

This year I got to try my hand at being on the blue team in an attack / defense style competition at BSidesLV 2016. First, a little about PvJ; Pros Vs Joes is a very interesting competition, as day 1 is purely blue team (defense) based, where you have to defend your network against a team of professional penetration testers (redteam). Day 2 is a little different because the red team disperses to help the blue teams attack each other (purple team time!). It’s a great CTF because it strives to not only put contestants in the hot seat, but to enable rapid growth and mentoring by pairing experienced players with inexperienced players. The competition creates an overwhelming amount of tasks to complete, meaning all resources have to be used to their maximum capability. There are a lot of great first-time Joe experiences out there, that really emphasize how impactful this pairing can be. This writeup is going to cover our games highlights, lowlights, and also attempt to provide some strategy for future contestants.

We put in a ton of preparation, from weekly meetings, high level strategy guides, hardening scripts, and even preparing auxiliary / special teams (network monitoring and attack teams). We compiled a number of cheat sheets and resources, as well as outlining scripts for the automated defense and offense of both Linux and Windows. We would have a weekly call where we set preparation milestones and contribute towards these in chat throughout the week. This preparation allowed us to accurately gauge people's involvement, skill level, and generate investment. The team really came together in this regard to make sure we had a well rounded / prepared strategy, where we asked many clarifying questions and amended our strategy to the answers from gold team (Dichotomy and admins). I know other teams put in similar preparation, and we’ve discussed distilling some of these notes and scripts into a high level wiki for all newcomers.

Before I dive into the actual game, there were a number of minor bugs with the scoreboard, as Dichotomy explains here, that ended up throwing off our calibration and what to prioritize for points, as is reflected in the scoreboard screenshots throughout the post. So when it looks like we are winning, we really aren’t lol. That said, I was pretty happy to see that we won day 1 when it was counted without any of the tickets, however the Day 2 score included these hidden ticket scores, and this sank us overall. Despite any error, it was an incredibly fun and competitive game, something I will certainly be playing again!! Scorebot was also a little funky in submitting flags, something we didn’t get until the very end of day one, but after that we were off to the races with submitting flags we found on machines. I plan to dev on scorebot (which is open source!) in the future, and there are already some great devs making that easier.

It was interesting to note that most of the infrastructure was pretty similar to last years. For example, the network ranges, infrastructure, and services were pretty similar when compared to previous writeups. This means experience in this game or even reading these past writeups can go a LONG way in aiding players both in strategy and technical preparedness. This is one game where despite the feeling of being thrown into the roil, preparation will pay off in droves. Our team, Team Machine, played a very strong network monitoring game, by utilizing Snort plugins in our PFSense firewall, which allowed us to quickly spot compromised hosts and figure out which machines we needed to dig on. This allowed us to catch a number of attacks against us, such as a DNS TKEY DOS, as well as post exploitation command and control channels, like Metasploit running on it’s default ports of 4444.

Throughout both days it was an incredibly competitive game, with lots of back and forth pwnage. Every team had services down, a compromised machine, compromised other machines, fended off attacks, and brought services back up by the end of competition. One of our biggest mistakes was not understanding our various web vulnerabilities, for example there were various exploits aside from default credentials and simple patching in these web apps.

One of my team's strongest factors was our ability to organize and collect flags on all of our machines, as well as any opponents machine we compromised. We kept track of all the flags we collected, as well as the location of the flags, this made it easy to locate them quickly when we compromised another machine. One thing I didn’t like is that at the end of the game it was possible to purchase raffle tickets and trade these for flags, which created a minor spending war between our team and SYNdicate. It’s important to note that the sale of these raffle tickets went directly to the charities of BSidesLV, like the EFF and Hackers for Charity, which was Dichotomies intent to support. Ultimately this spending war had little effect on the final score and was a funny twist.

One of our biggest security weakness was vulnerable web applications and web shell post exploitation. This is how our PBX got owned the second day, as well as a few other machines. Having the phones was a huge advantage, so we were persistent on bringing it back, but didn’t notice the webshell getting activated in our access logs, as it was hidden in all of the web application scanning. That said, we have some secret tactics prepared to deal with this for next time, things I learned from the other pros after the game. 

Ultimately, our downfall was not monitoring our IT tickets close enough and dealing with those tasks appropriately. We were receiving requests both via an ingame ticketing system and via ingame emails. Oddly enough, we could view all other teams tickets in the ticketing system as well, which we used to scrape usernames and passwords that they were provisioning for the grey cell (user emulation). We used this for new access and privilege escalated later, but not paying attention to our own tickets and IT tasks was a major drag on points, which past teams have also pointed out. You can see in my scoreboard screenshots that this really threw off our calibration, as we thought we were doing well here when in fact we weren’t doing so well.

The winning team was team SYNdicate, captained by my friend and Shadow Cats teammate Matir. He has extensive experience playing this game, and his team’s win was well earned and honestly can’t be contested despite any scoring issues, they played a strong game, and it was a ton of fun. We also had a pretty candid discussion after over dinner, where we both revealed our hands and tactics in the spirit of the PvJ learning environment. It was an incredible experience and really reflects how the game is about learning and getting better, not about beating people and not explaining how it happened. Finally, Dichotomy asked me to be a future Blue Team Captain, a role I accepted as I want to see this game grow and flourish, as it’s both incredibly fun and educational.