From Novice to Ninja: Proxy Techniques in Pentesting
13 min read
March 24, 2024
Table of contents
Introduction: Navigating the Web Penetration Testing Seas
Welcome aboard, digital explorers! You're about to embark on an extraordinary voyage through the treacherous yet thrilling waters of web penetration testing. In this realm, proxies are more than mere tools; they are our compass and map, guiding us through the murky depths of web applications to uncover hidden treasures and lurking dangers alike.
Proxies, the unsung heroes of the cybersecurity world, offer us a unique vantage point from which to observe, intercept, and manipulate the data flowing between our browsers and the vast expanse of the internet. They are the linchpins in our quest to fortify our digital fortresses against the relentless onslaught of cyber threats.
In this chapter, we'll be charting a course through the intricate network of proxies, with a special focus on mitmproxy, a versatile and powerful ally in our penetration testing arsenal. Whether you're a seasoned sailor of the cyber seas or setting foot on the deck for the first time, you'll find valuable insights and strategies to enhance your web security endeavors.
Setting Sail with Proxies
Our journey begins with an exploration of the diverse species of proxies that inhabit the digital ocean. From the shadowy realms of Anonymous Proxies to the bustling trade routes guarded by Reverse Proxies, we'll navigate the complexities of these vital tools. Understanding their unique characteristics and applications will equip us with the knowledge to choose the right proxy for each mission.
Arming Ourselves for Adventure
With our bearings set, we'll delve into the arsenal of tools at our disposal, with a spotlight on mitmproxy. This lightweight yet formidable tool will be our primary instrument in dissecting and defending web applications. Through hands-on guidance and practical examples, we'll master the art of using mitmproxy to unveil vulnerabilities and safeguard our digital domains.
The Craft of Configuration
Before diving into the action, we'll lay the groundwork by configuring our environment. Setting up mitmproxy and integrating it with our browsers and systems is akin to tuning our instruments before a concert—a step essential for the symphony of security testing to proceed harmoniously.
As we sail towards the horizon, we'll engage in real-world scenarios, employing our proxy to intercept, examine, and manipulate web traffic. From the bustling marketplaces of OWASP Juice Shop to the hidden coves of custom applications, we'll apply our skills in live environments, bridging the gap between theory and practice.
Enriching Our Voyage with Scripts
Our adventure is further enhanced by the magic of scripts, extending mitmproxy's capabilities to suit our every need. Whether it's exporting requests for further analysis or automating sophisticated attack sequences, we'll discover how to tailor our toolkit to become the ultimate web penetration testing craftsman.
As we chart this course together, our journey will be illuminated by the twin stars of curiosity and diligence. With each challenge conquered and mystery unraveled, we'll grow not just as testers but as guardians of the digital realm.
So, fasten your seatbelts and prepare your tools—our voyage into the world of web penetration testing and proxies is about to begin. The seas may be rough and the challenges daunting, but the rewards of securing our digital horizons are unparalleled. Welcome to the adventure of a lifetime.
Types of Proxies
In the vast ocean of proxies, there are several species, each adapted to its unique environment. Let's meet some of the most common ones:
- Anonymous Proxies: The ninjas of the proxy world. They hide your IP address, making your online activities invisible to prying eyes. Perfect for testers who need to operate without leaving a trace.
- Transparent Proxies: These guys don't hide your IP address but still forward your requests. They're like the honest workers of the internet, often used for caching web pages and controlling employee internet usage.
- Reverse Proxies: The gatekeepers. They sit in front of web servers, directing incoming traffic to the correct destination. They're great for balancing loads and providing additional security layers.
- HTTP vs. SOCKS Proxies: The classic debate. HTTP proxies understand and interpret web traffic, making them ideal for viewing and modifying web pages. SOCKS proxies, on the other hand, are like the versatile adventurers—they handle all types of traffic, not just web pages, making them a go-to for testers working with various applications.
Each type of proxy has its stage and script in the theater of web penetration testing. Choosing the right proxy is like selecting the right tool for a job—it can make all the difference in how effectively you can test and secure a web application.
Setting Up and Tooling Up for Success
Now that we've navigated the waters of proxy types, it's time to gear up and get our hands dirty. Setting up a proxy for web penetration testing might sound like rocket science, but fear not! It's more like assembling a high-tech LEGO set. Let's talk about the toolkit that's going to make you the master builder of web security.
Burp Suite
First up is the legendary Burp Suite, a favorite among web security professionals. Think of it as your digital Swiss Army knife, equipped with everything from basic mapping to advanced vulnerability exploitation tools. It's like having a superhero sidekick, ready to leap into action, whether you're just getting started or diving deep into the vulnerabilities of a web application.
OWASP ZAP
Next, we have OWASP ZAP (Zed Attack Proxy), the open-source wonder. It's the Robin to your Batman, offering a powerful range of tools to detect vulnerabilities in web applications. With its user-friendly interface and community-backed support, ZAP makes sure you're well-equipped to tackle security threats head-on.
Mitmproxy
Last but definitely not least, meet mitmproxy, the tool we'll be getting up close and personal with throughout our series. This lightweight proxy is like the agile spy of the internet, capable of intercepting, inspecting, modifying, and replaying web traffic. Its console interface might remind you of piloting a spaceship with text commands, but once you get the hang of it, you'll be maneuvering through web traffic like a pro. We're going to spend a lot of time with mitmproxy, uncovering the secrets of web applications and learning how to shield them from potential threats. So buckle up, it's going to be an exciting ride!
Setting the Stage
Before we jump into action, let's set the stage. Configuring your proxy involves a few key steps: installing the software, setting your browser to route traffic through the proxy, and ensuring your proxy is listening to the right port. It's like tuning your musical instrument before a concert; everything needs to be pitch-perfect for the performance ahead.
In this case, I am going to show you how to do it with mitmproxy, but if you are using another one like burp, the steps are very similar, in the end you just need to create a trusted certificate and install it in your favorite browser.
Anyway, the different steps can be found in the official documentation:
First off, we need to set up FoxyProxy on our browser to listen on the right port. I’ve chosen port 8888 for this demonstration. This step is like telling FoxyProxy where to direct the web traffic for our investigative purposes.
With FoxyProxy configured, it’s time to bring mitmproxy into play. Open your terminal and enter the magic incantation:
mitmproxy -p 8888
This command starts mitmproxy and tells it to keep an eye on port 8888, our chosen gateway to the web's hidden secrets.
Navigate to "mitm.it" using the browser where you’ve set up FoxyProxy. Here, you'll be greeted by a dashboard offering various certificates. Download the one for Linux (or your respective operating system). Next up, we’re heading to browser settings to make it trust our new friend, the mitmproxy certificate.
Installing the Certificate in Firefox (Chrome users, your journey is similar):
- Open Firefox settings and head over to the ‘View Certificates’ section.
- Click on ‘Import’ and follow the prompts, ticking all the right boxes to secure Firefox’s trust in our certificate.
With the certificate installed, visiting the OWASP Juice Shop (or any web application you're testing) should unveil all its web traffic in mitmproxy's console, like lifting the curtain on a stage.
To streamline your web testing adventures, consider creating a configuration file for mitmproxy. This little cheat sheet automatically sets your preferred settings, like the port number and any specific scripts you want to run (we’ll dive into the power of scripts later).
Here’s a pro tip: Take a moment to explore mitmproxy’s tutorials. They’re like a treasure map to mastering this tool, guiding you through basic to advanced maneuvers. And don’t worry if you see a lot of Vim-style shortcuts; if you’ve ventured through the realms of Vim before, you’ll feel right at home.
Why mitmproxy?
You might wonder why we're spotlighting mitmproxy in our series. It's simple: mitmproxy offers a blend of simplicity, power, and flexibility unmatched by many other tools. Whether you're a rookie or a seasoned pro, mitmproxy scales with your skills, offering deep insights into web traffic and vulnerabilities. Plus, its open-source nature means you're joining a community of experts dedicated to making the web a safer place.
Practical Use Cases in Penetration Testing
Entering the heart of our adventure, it's time to explore the real-world application of proxies in web penetration testing. Armed with our tools, especially mitmproxy, we're ready to tackle the challenges that lie ahead. In this section, we'll outline various scenarios where proxies become the linchpin of our testing strategy, demonstrating their value in uncovering and exploiting vulnerabilities in web applications.
Interception and Modification of Requests and Responses
One of the core strengths of using proxies like mitmproxy is the ability to intercept and modify HTTP/HTTPS requests and responses in real-time. This capability allows us to:
- Inspect Headers and Cookies: By examining the headers and cookies, we can uncover security flaws such as insecure settings or vulnerable tokens.
- Manipulate Data: Altering inputs and outputs enables us to test the application's handling of unexpected or malicious data, identifying potential input validation and sanitization issues.
Analyzing Encrypted Traffic
With the majority of web applications using HTTPS for security, it's crucial to be able to analyze encrypted traffic. Proxies allow us to decrypt and inspect this traffic, which is essential for:
- Identifying Sensitive Data Exposure: Revealing how an application encrypts and transmits sensitive information can highlight potential data leakage points.
- Testing SSL/TLS Configuration: Ensuring that the application's encryption settings are up to snuff and not vulnerable to attacks like SSL stripping or vulnerable cipher suites.
Automating Custom Attacks
Proxies not only let us perform manual testing but also automate custom attacks. This is particularly useful for:
- Replaying and Customizing Attacks: Once we identify a request that leads to a vulnerability, we can replay it with modifications to explore the depth and impact of the issue.
- Crafting Sophisticated Attack Sequences: By automating attacks, we can simulate complex attack sequences that would be time-consuming or difficult to perform manually.
Identifying Logic Flaws and Session Management Issues
Logic flaws and session management vulnerabilities can be subtle and hard to detect. Using proxies, testers can:
- Session Hijacking and Fixation: Intercepting and modifying session tokens can reveal vulnerabilities in how sessions are managed and protected.
- Testing Access Controls: By modifying user roles or permissions in requests, we can test the effectiveness of access control mechanisms.
Real-World Application: Juice Shop
Let's dive into the practical arena with our case study, the Juice Shop. This section will walk you through how I leveraged proxies, specifically mitmproxy, to unearth vulnerabilities and perform tests that echo real-world scenarios. Through hands-on examples from Juice Shop, we'll see the pivotal role proxies play in bridging theory with the gritty reality of web security.
One fascinating experiment to conduct with an application like Juice Shop involves exploring whether a user can leave comments while masquerading as another user. This test shines a light on how the application manages comment functionalities and the potential loopholes therein.
Setting Up the Stage
First things first, let’s get our proxy, mitmproxy, into interception mode. Depending on where your testing environment is hosted, you'll adjust the viewing filter accordingly. For those working with a remote setup, you might use a command similar to:
set view_filter '192.168.20.120'
If you’re tinkering directly on your local machine, localhost will be your target.
The Test Run
With mitmproxy lying in wait, let's initiate a comment post to understand the mechanics behind it. Intercepting the request unveils that both the author's name and the message content are neatly packed in the request body, ripe for modification.
To edit the request, a simple dance of pressing "e" for edit and "a" for accessing the request body gets us to the editor set by your $EDITOR environment variable. Here, in my playground, I decided to switch the author name to impersonate another user.
After tweaking the request to our liking and pressing "a" to send it off, mitmproxy confirms our mischief has been successfully executed by showing the altered comment now attributed to a different user.
Watching the Dominoes Fall
To wrap up our little experiment:
set view_filter ''
Clearing the view filter allows us to witness the outcome in the product reviews section—our comment, now falsely attributed to another user, standing as a testament to the discovered vulnerability.
The Takeaway
This scenario offers just a glimpse into the utility of proxies in a security audit. By scrutinizing feature behaviors and toying with parameters, we can uncover oversights not immediately apparent or reachable by automated scanners like Burp Suite or Nuclei. The nuanced vulnerabilities we manually unravel through proxies can be both challenging and rewarding, underlining the importance of mastering this tool in your security toolkit.
Enhancing mitmproxy with Scripts
One of the most exhilarating aspects of mitmproxy lies in its versatility, thanks to its plugin system. This feature is a game-changer, enabling you to tailor the tool to your heart's desire. Consider this: with just a few lines of code, you can forge a command named "exporter" that sieves through your marked requests, exporting them in their raw glory to a specified directory.
The Exporter Plugin
from mitmproxy import ctx, http
from mitmproxy import command, types, flow
import os
class Exporter:
EXPORT_DIR = os.getcwd() + "/raw_requests"
def format_request(self, flow: http.HTTPFlow) -> str:
request_line = f"{flow.request.method} {flow.request.pretty_url} {flow.request.http_version}"
headers = '\n'.join(f"{name}: {value}" for name, value in flow.request.headers.items())
body = flow.request.get_text(strict=False)
return f"{request_line}\n{headers}\n\n{body}"
@command.command("exporter.export_marked")
def export_marked(self, flows: types.Sequence[flow.Flow], dir: str = EXPORT_DIR) -> None:
if not os.path.exists(dir):
os.makedirs(dir)
for flow in flows:
if flow.marked:
filename = f"{flow.request.host}-{flow.request.method}-{int(flow.request.timestamp_start)}.txt"
filepath = os.path.join(dir, filename)
raw_request = self.format_request(flow)
with open(filepath, "w") as file:
file.write(raw_request)
ctx.log.info(f"Exported request: {filepath}")
addons = [
Exporter()
]
To deploy this script, simply append it to mitmproxy with the -s flag, like so:
mitmproxy -s requests_dumper.py
Or, integrate it into your configuration file for a seamless experience.
Practical Magic with Commands
Imagine you’re dissecting a login sequence and wish to archive its requests for further analysis or fuzzing. Mark the relevant requests and unleash your custom command to funnel them into the "requests" directory, ready for any operation you plan next.
A Twist with mitmdump
Mitmdump, mitmproxy’s CLI sibling, offers a parallel path. It stands guard, diligently archiving all your navigational requests into the "raw_requests" directory.
from mitmproxy import http
import os
EXPORT_DIR = "raw_requests"
def ensure_dir_exists(directory):
if not os.path.exists(directory):
os.makedirs(directory)
def request(flow: http.HTTPFlow):
ensure_dir_exists(EXPORT_DIR)
# Construye un nombre de archivo basado en detalles de la solicitud
filename = f"{flow.request.host}-{flow.request.method}-{int(flow.request.timestamp_start)}.txt"
filepath = os.path.join(EXPORT_DIR, filename)
# Formatea la solicitud en el formato raw especificado
request_line = f"{flow.request.method} {flow.request.pretty_url} {flow.request.http_version}"
headers = '\n'.join(f"{name}: {value}" for name, value in flow.request.headers.items())
body = flow.request.get_text(strict=False)
raw_request = f"{request_line}\n{headers}\n\n{body}"
# Guarda la solicitud formateada en el archivo
with open(filepath, "w") as file:
file.write(raw_request)
Activate it with a -s flag:
mitmdump -s ~/Tools/mitmproxy/raw_dumper.py
This is but a peek into the realm of plugins and scripts with mitmproxy. Imagine coupling this with tools like ffuf or katana, orchestrating a symphony of tests from the comfort of your proxy. The possibilities? Limitless.
Conclusion: Charting the Course Forward
As we dock back into the harbor after our voyage across the complex seas of web penetration testing, it's time to reflect on the journey we've undertaken together. Through the mists of technical challenges and the storms of cybersecurity threats, proxies have been our steadfast allies, illuminating the path to understanding and securing web applications.
The Power of Proxies Unleashed
Our exploration revealed the critical role proxies like mitmproxy play in the arsenal of a web penetration tester. By serving as the lens through which we can inspect, intercept, and manipulate web traffic, proxies empower us to uncover vulnerabilities that could otherwise lay hidden beneath the surface. They are not just tools but extensions of our will to protect and secure the digital domain.
Mastery Through Practice
The hands-on examples, from setting up our environment to delving into the inner workings of applications like Juice Shop, demonstrated the practical application of proxies in real-world scenarios. These exercises underscored a fundamental truth: mastery in web penetration testing comes through practice, experimentation, and a relentless pursuit of knowledge.
In Closing
Our voyage through the realm of proxies and web penetration testing may have reached its conclusion, but the adventure of learning and exploration never truly ends. Armed with the knowledge and skills you've acquired, you are now a beacon of security in the digital age, a protector of the vast web applications that power our world.
Remember, the sea of cybersecurity is wide and deep, filled with both peril and promise. It's up to us, the navigators of this digital world, to chart a course toward a safer future. So, keep your compass true, your maps updated, and your curiosity alive. The next horizon is yours to explore.
Chapters
Previous chapter
Next chapter
