From Chaos to Clarity: The Art of Fuzzing with Nuclei

13 min read

April 18, 2024

Open Source Web Hacking Mastery: A Junior's Guide to Methodical Penetration Testing

From Chaos to Clarity: The Art of Fuzzing with Nuclei

Introduction

In the vast and ever-evolving universe of cybersecurity, where digital threats constantly morph and lurk in the shadows, I embark on a personal crusade to fortify our digital defenses. This installment of my blog marks the beginning of an exhilarating venture: the creation of a personal repository of custom Nuclei templates. Each template, crafted from my experiences and insights, is a key designed to unlock the mysteries of unseen vulnerabilities through the art of fuzzing. Today, we zero in on developing a specialized template to sniff out SQL Injection (SQLi) vulnerabilities in POST requests, venturing beyond mere vulnerability scanning to chart unexplored territories with Nuclei and fuzzing. Here are the main takeaways from our upcoming exploration:

Advancing Cybersecurity with Nuclei: Leveraging Nuclei's robust framework to develop custom templates that enhance our ability to detect and analyze vulnerabilities. This approach underscores the importance of precision and adaptability in our defense mechanisms.
The Technical Core of SQLi Detection: Exploring the technical nuances of crafting a Nuclei template specifically designed to detect SQLi vulnerabilities in POST requests. This includes a detailed examination of payload construction, filtering conditions, and the strategic implementation of fuzzing techniques to identify vulnerabilities that evade conventional detection methods.
Integrating mitmproxy for Enhanced Testing: Utilizing mitmproxy, a versatile tool for intercepting, inspecting, modifying, and replaying HTTP traffic, to capture and format requests in a Nuclei-compatible format. This step is crucial for setting up accurate and effective testing environments, demonstrating the synergy between various cybersecurity tools in identifying and mitigating vulnerabilities.

Join me as we delve into the nuances of crafting a Nuclei template, an exploration that is more than a tutorial—it's a testament to the commitment to bolstering our digital fortifications, one template at a time. Together, let's navigate this journey towards a more secure digital future.

Nuclei and Fuzzing: A Quick Refresh

Remember our adventures through the digital landscape in previous chapters? We took a deep dive into the world of Nuclei in one thrilling episode, exploring how this ingenious tool acts like a digital Sherlock Holmes, meticulously sifting through web applications and networks with its template-based detection. Then, we embarked on a wild ride with fuzzing in our exploration of ffuf, where we embraced the chaos of sending a barrage of unexpected data at applications just to see what breaks. It's time to connect those dots.

Revisiting Nuclei: Cast your mind back to when we first met Nuclei. We touched on its ability to use predefined templates for vulnerability detection, making it an indispensable tool for cybersecurity professionals looking to write their own detective stories. Yes, Nuclei is that specialized scanner from Project Discovery that allows for such tailored investigations, enabling us to target specific vulnerabilities with precision. Think of it as having a highly customized magnifying glass to spot those digital clues.

Fuzzing - The Art of Digital Chaos, Revisited: And who could forget our venture into the world of fuzzing with ffuf? We learned that fuzzing isn’t just about creating a mess; it’s a calculated strategy to unearth vulnerabilities that are so well-hidden they’re practically incognito. By challenging software with the unexpected—inputs that defy the norm—we push systems to their limits, revealing weaknesses that standard testing might miss.

As we gear up to dive deeper into leveraging Nuclei with custom templates for fuzzing, let’s not forget the ground we’ve covered. We've seen the precision of Nuclei and the controlled chaos of fuzzing with ffuf. Now, it’s time to see how these methodologies can be combined and elevated. Our journey into the advanced use of Nuclei for discovering previously undetected vulnerabilities begins here. Buckle up; it’s going to be an insightful ride.

Advanced Fuzzing with Custom Nuclei Templates

In the realm of cybersecurity, Nuclei stands out for its capacity to execute advanced fuzzing via custom templates. By fully leveraging Nuclei’s capabilities, security enthusiasts are equipped to conduct precise and adaptable penetration testing, uncovering vulnerabilities that have eluded detection. Let’s delve into the critical elements that endow Nuclei with such prowess in fuzzing, followed by an illustrative example of a template.

The Essentials of Crafting Nuclei Fuzzing Templates

Supported Fuzzing Parts: Nuclei templates offer the versatility to target specific segments of an HTTP request, enhancing the tool’s applicability across various fuzzing scenarios. The parts that can be targeted include:

Query: For the manipulation of query parameters.
Path: To alter URL paths.
Header: For modifying request headers.
Cookie: To adjust cookie values.
Body: Tailoring the request body's content, compatible with formats such as JSON, XML, Form, and multipart-form data.

This granularity enables the simulation of a wide array of attack vectors, enriching the fuzzing landscape.

Rule Types: To cater to diverse fuzzing needs, Nuclei supports a plethora of rule types for payload insertion:

Prefix: Prepending a payload before the value.
Postfix: Appending a payload after the value.
Replace: Substituting the value with a payload.
Infix: Embedding a payload within the value.
Replace-regex: Leveraging regex for sophisticated replacement operations.

These rule types provide the flexibility necessary to craft templates that can probe for an extensive range of vulnerabilities.

Loading HTTP Traffic: Nuclei’s ability to import HTTP traffic from various sources significantly broadens its testing scope:

Compatibility with tools such as Burp Suite, httpx, and Proxify.
Support for API schema files like OpenAPI, Swagger, and Postman collections.

These integrations ensure that fuzzing efforts closely mirror real-world traffic patterns, increasing the likelihood of uncovering critical security issues.

Filters: A key aspect of refining the fuzzing process is the use of filters within Nuclei templates. Filters allow for the specification of conditions under which a template is activated, based on the characteristics of the HTTP request. This targeted approach aids in minimizing unnecessary noise and focusing efforts on potentially vulnerable areas.

Efficiency Through Abstraction: Nuclei enhances template creation efficiency by abstracting the parts of the HTTP request into key-value pairs. This abstraction simplifies the application of fuzzing rules across different data formats, enabling broader testing capabilities with fewer templates.

Illustrative Template Example: Header Manipulation for Security Testing

Below is a detailed Nuclei template designed to test the security robustness of web applications by manipulating HTTP headers. This template specifically aims to identify vulnerabilities that could be exploited via header injection attacks, a technique often used in web application attacks such as HTTP Request Smuggling or Web Cache Poisoning:

http:
  - pre-condition:
      - type: dsl
        dsl:
          - 'method == "POST"'       # only run if method is POST
          - 'contains(path,"reset")' # only run if path contains the word "reset"
        condition: and
    # fuzzing rules
    fuzzing:
      - part: header # This rule will be applied to the header
        type: replace # replace type of rule (i.e., existing values will be replaced with payload)
        mode: multiple # multiple mode (i.e., all existing values will be replaced/used at once)
        fuzz:
          X-Forwarded-For: "{{domain}}"  # here {{domain}} is an attacker-controlled server
          X-Forwarded-Host: "{{domain}}"
          Forwarded: "{{domain}}"
          X-Real-IP: "{{domain}}"
          X-Original-URL: "{{domain}}"
          X-Rewrite-URL: "{{domain}}"
          Host: "{{domain}}"

Template Insights:

Pre-Condition: The template is triggered based on pre-defined conditions, ensuring it's executed for POST requests that include "reset" in the path. This specificity helps focus the fuzzing efforts on potentially vulnerable endpoints involved in sensitive operations like password resets.
Fuzzing Rules: The core of this template lies in its fuzzing rules, where it replaces existing header values with payloads that include a domain controlled by the attacker. This method tests the application's handling of headers and whether it's possible to manipulate the application's behavior or leak sensitive information by injecting malicious header values.
Mode: By setting the mode to "multiple," the template ensures that all specified headers are modified in a single request, allowing for comprehensive testing of the application’s response to multiple, simultaneous header manipulations.

Methodology for creating templates

Beginning the Template Creation Process

To kick off the template creation, we first need to capture a typical request, such as a login attempt. This task can be accomplished using any proxy tool; in this scenario, I've opted for mitmproxy for its ease of use and versatility.

Handling GET vs. POST Requests with Nuclei

Auditing GET requests with Nuclei is fairly straightforward: we simply input the URL and, if necessary, provide authentication cookies for testing. However, auditing POST requests requires a more tailored approach due to the specific format needed for Nuclei analysis. To audit effectively with Nuclei, our captured request must be in one of the following formats: Burp, JSONL, YAML, OpenAPI, or Swagger. In this context, we'll demonstrate how to convert a request into the JSONL format using a command in mitmproxy. Should you be utilizing Burp, you have the convenience of directly copying a request in the required format.

Transforming Requests into Nuclei-Compatible Format

Below is a Python snippet that defines a mitmproxy command to log URLs and, where relevant, the bodies of POST requests in a JSONL format compatible with Nuclei's requirements:

💡

If you want to know more about mounting commands, I recommend you to take a look at the chapter where we talk about mitmproxy.

    @command.command("hacking.log_urls_in_nuclei_format_with_body")
    def log_urls_in_nuclei_format_with_body(self, flows: types.Sequence[flow.Flow]) -> None:
        """
        Logs URLs and, if applicable, POST request bodies in a JSONL format that aligns with Nuclei's expectations.
        """
        export_dir = os.path.join(os.getcwd(), "nuclei_formatted_urls_with_body")
        os.makedirs(export_dir, exist_ok=True)
        
        filename = "nuclei_formatted_urls_with_body.jsonl"
        filepath = os.path.join(export_dir, filename)

        with open(filepath, "a") as file:
            for flow in flows:
                # Simplify the header and include body if the method is POST
                method = flow.request.method
                body = flow.request.get_text(strict=False) if method == "POST" else ""
                headers = {name: value for name, value in flow.request.headers.items()}
                
                # Construct the raw request string, including the body for POST requests
                raw_request_lines = [
                    f"{method} {flow.request.path} HTTP/1.1",
                    *[f"{name}: {value}" for name, value in headers.items()],
                    "",  # End of headers
                    body
                ]
                raw_request = "\r\n".join(raw_request_lines)

                # Create the entry to be logged
                nuclei_entry = {
                    "timestamp": datetime.now(timezone.utc).isoformat(),
                    "url": flow.request.pretty_url,
                    "request": {
                        "header": headers,
                        "raw": raw_request
                    },
                    # Omitting the response part in this example
                }

                file.write(json.dumps(nuclei_entry) + "\n")
        
        ctx.log.info(f"URLs and POST bodies logged in Nuclei-compatible format: {filepath}")

Utilizing the Command in Mitmproxy

To apply the command in mitmproxy and convert a selected request into JSONL format, you can use the following command:

hacking.log_urls_in_nuclei_format_with_body @focus

This streamlined process allows us to efficiently capture and format requests, paving the way for thorough vulnerability assessments with Nuclei.

Creating the template

To begin with, I have taken a small template from one of the resources at the end of the article. The content is as follows:

http:
    # filter checks if the template should be executed on a given request
  - filters:
      - type: dsl
        dsl:
          - method == POST
          - len(body) > 0
        condition: and
    # payloads that will be used in fuzzing
    payloads:
      injection: # Variable name for payload
        - "'"
        - "\""
        - ";"
    # fuzzing rules
    fuzzing:
      - part: body  # This rule will be applied to the Body
        type: postfix # postfix type of rule (i.e., payload will be added at the end of exiting value)
        mode: single  # single mode (i.e., existing values will be replaced one at a time)
        fuzz:         # format of payload to be injected
          - '{{injection}}' # here, we are directly using the value of the injection variable

Unpacking the Nuclei Template

Imagine you’re a detective, and your job is to find the hidden doors and weak spots in a building (in our case, a web application). A Nuclei template is your map and toolkit rolled into one, guiding you where to look and what tools to use.

The Blueprint: Filters

First up, we have the filters. Think of this as your checklist before you embark on your investigation. The template specifies two main conditions under this section:

The method of communication must be a POST request, akin to sending a parcel rather than just a letter, requiring a response from the recipient.
The parcel (the body of the request) cannot be empty; it must contain something to probe the application's reaction.

The rule here is straightforward: both conditions must be met (condition: and) for our investigative tool to spring into action.

The Toolkit: Payloads

Next, we delve into the payloads section. Here, we find an array of peculiar phrases or symbols - in our case, a single quote ('), a double quote ("), and a semicolon (;). These might seem innocuous at first glance, but they're akin to picking locks in the digital world. Each one is a test to see if we can trip up the application's normal processing and reveal hidden vulnerabilities.

The Strategy: Fuzzing Rules

Finally, the fuzzing section lays out our strategy for using these tools. Here’s the breakdown:

We’re targeting the body of the request, the main content of our digital parcel.
We employ a postfix approach, adding our lock-picking tools to the end of the existing message, tweaking the message's tail to see if it unlocks any reactions.
Our mode of operation is single, meaning we test one tool at a time, giving each its moment to shine or fail, ensuring we can pinpoint exactly which tool reveals a vulnerability.
The fuzz key tells us we’re directly using our array of tools (injection payloads) in this operation, applying them precisely as outlined in our toolkit.

Launching the Template

For execution, metadata must precede the template. Here's an example:

id: testing-stored-xss
info:
  name: SQLi POST request
  author: rsgbengi
  severity: critical
  tags: sqli,das

Executing the template to detect vulnerabilities is straightforward. However, enabling debug mode and using a proxy for manual request testing is advisable:

nuclei -im jsonl -l nuclei_formatted_urls_with_body/nuclei_formatted_urls_with_body.jsonl  -debug -t sqli-post.yaml -proxy http://127.0.0.1:8888

Despite encountering errors related to "filters" in my current Nuclei version, I've improvised by integrating a matcher designed to identify specific keywords, albeit with a potential for false positives:

  - matchers:
      - type: dsl
        dsl:
          - method == POST
          - len(body) > 0
        condition: and
      - type: word
        name: node
        words:
          - "Error"
          - "SQL"

To streamline testing, I've employed a tag to cease execution upon the first match, avoiding exhaustive combinatorial testing:

    stop-at-first-match: true

The complete template I am going to use can be found here:

id: testing-sqli
info:
  name: SQLi POST request
  author: rsgbengi
  severity: critical
  tags: sqli,dast
http:
  - matchers:
      - type: dsl
        dsl:
          - method == POST
          - len(body) > 0
        condition: and
      - type: word
        name: node
        words:
          - "Error"
          - "SQL"
    stop-at-first-match: true
    payloads:
      injection: 
        - "'"
        - "\""
        - ";"
    fuzzing:
      - part: body  
        type: postfix 
        mode: single  
        fuzz:         
          - '{{injection}}'

If we run now with the changes made, we can see how we can indeed detect the SQLI vulnerability thanks to the word "Error".

Identification of possible injection from "Error"

The Refined Template

Enhancing our template to reduce false positives involves incorporating more selective error messages:

id: refined-testing-sqli
info:
  name: Refined SQLi POST Request Detection
  author: rsgbengi (refined)
  severity: critical
  tags: sqli,dast
http:
  - method: POST
    matchers:
      - type: dsl
        dsl:
          - "method == 'POST'"
          - "len(body) > 0"
        condition: and
      - type: regex
        name: sql_error
        regex:
          - "You have an error in your SQL syntax;"
          - "Warning: mysql_fetch_assoc()"
          - "Unclosed quotation mark after the character string"
          - "ORA-[0-9]{5}"
        condition: or
      - type: status
        status:
          - 500
          - 200
    payloads:
      injection:
        - "' OR '1'='1"
        - "1' WAITFOR DELAY '0:0:5' --"
        - "' EXEC xp_cmdshell('dir') --"
    fuzzing:
      - part: body
        type: postfix
        mode: single
        fuzz:
          - "{{injection}}"
    stop-at-first-match: true

Detection of vulnerability from error 500

In any case, you can use for example for sqli, the regex that come in this template and add more or less words as you find them:

On the other hand, for quick template creation, there is a burp plugin that automates a large part of the process. Here is the link in case you want to take a look at it:

Conclusions

As we wrap up this chapter of our cybersecurity odyssey, it's clear that the fusion of Nuclei's targeted precision with the dynamic unpredictability of fuzzing opens new frontiers in the battle against digital vulnerabilities. The journey we embarked on today—starting with the crafting of a custom Nuclei template for detecting SQL Injection in POST requests—serves as a cornerstone in our ongoing quest to safeguard the digital realm.

This personal endeavor to build a repository of Nuclei templates is not just about enhancing our defensive arsenal; it's a testament to the power of collective wisdom and individual initiative in the cybersecurity community. By sharing these insights and tools, we not only strengthen our own defenses but also contribute to the broader network of digital protectors.

I encourage you to dive deeper, to explore the intricacies of Nuclei and the art of fuzzing further. The resources section below is a treasure trove of knowledge, meticulously curated to guide your journey through the complex tapestry of cybersecurity. Whether you're looking to refine your skills, seeking inspiration for your next project, or simply curious about the latest in vulnerability scanning and detection, these resources are your gateway to a wealth of information.

Let's continue to learn, to experiment, and to share. The path to mastering cybersecurity is an ever-evolving journey, filled with challenges and triumphs. By exploring the resources provided, engaging with the community, and contributing our unique insights, we not only defend against the threats of today but also prepare for the unknown challenges of tomorrow.

Thank you for joining me on this adventure. Together, let's forge ahead, armed with knowledge and innovation, in our relentless pursuit of a secure digital future.