Palo Alto Networks Minemeld - Part III - Additional Miners

This post elaborates upon the previous previous posts in this series. If you haven't read through parts 1 and 2, I highly recommend that you start there prior to moving forward.

NOTE: The post(s) associated with this topic assume(s) that you already have an active Minemeld instance running (either open source or via AutoFocus), and that you already have a firewall or third party security tool configured in a manner that can consume the data. More information about initial configuration can be found here (for hosted) and here (for open source).

The purpose of this post is to build onto the previous configuration to further illustrate how Minemeld can be used to prevent threats using a wide variety of data sources. Your Minemeld instance should currently look like this (navigate to Config):

The additional data sources used below are just examples, but could be considered a good baseline with which to start. Feel free to choose others that may be more applicable to your specific deployment. Add the following miners to the configuration by navigating to Prototypes and selecting the Clone option:

• alienvault.reputation
• bambenekconsulting.c2_dommasterlist_high
• bambenekconsulting.c2_dommasterlist
• bambenekconsulting.c2_ipmasterlist_high
• bambenekconsulting.c2_ipmasterlist
• blocklist_de.all
• dshield.block
• ETOpen.blockIPs
• feodotracker.badips
• feodotracker.ipblocklist
• itcertpa.DOMAINS
• itcertpa.IP
• itcertpa.URLS
• libraesva.LIBRAESVA_Malware_Domains
• phishme.Intelligence
• proofpoint.EmergingThreatsDomains
• proofpoint.EmergingThreatsIPs
• ransomwaretracker.RW_DOMBL
• ransomwaretracker.RW_IPBL
• ransomwaretracker.RW_URLBL
• recordedfuture.DomainRiskList
• recordedfuture.IPRiskList
• spamhaus.DROP
• spamhaus.EDROP
• usom.DOMAINs
• usom.IPs
• usom.URLs
• zeustracker.badips

The Config will look like the following:

Add the following processors to the configuration by navigating to Prototypes and selecting the Clone option:

• stdlib.aggregatorDomain
• stdlib.aggregatorURL

The Config will look like the following:

Add the following output nodes to the configuration by navigating to Prototypes and selecting the Clone option (note that you will do this multiple times based on the output formats required):

• stdlib.feedHCGreen
• stdlib.feedMCGreen

The Config will look like the following:

For each processor node in the Config, add all relevant miners as inputs. The Config will look like the following:

For each output node in the config, add all relevant processor nodes as inputs. The Config will look like the following:

Upon commiting the Config, your Minemeld instance should look like this (navigate to Prototypes -> alienvault_reputation -> GRAPH):

As shown above, all miners are now sending data to their respective processor nodes, and based on confidence level value, being added to either a high or medium output node. Note, you will likely see a red exclamation mark next to some miners under the Nodes tab. This is because some of these miners require an API key, user name and password, etc. in order to gain access to the data. 

Palo Alto Networks Minemeld - Part II - Custom Miner Configuration

The purpose of this post is to elaborate on the configuration used to walk you through how Minemeld works in my previous post. If you haven't read through part 1, I highly recommend that you start there prior to moving forward.

NOTE: The post(s) associated with this topic assume(s) that you already have an active Minemeld instance running (either open source or via AutoFocus), and that you already have a firewall or third party security tool configured in a manner that can consume the data. More information about initial configuration can be found here (for hosted) and here (for open source).

Configuration Steps

1. Let's start fresh by deleting existing nodes under Config.
2. Navigate to Prototypes, which has its own tab in the AutoFocus-hosted version. You can get there in the open source version by navigating to Config -> Browse Prototypes (three lines in lower right-hand corner).
3. Search for and select ETOpen.blockIPs. 
4. Note that there are two options, New and Clone. We are going to select New.
1. New - this option allows you to take the prototype and create another one from it.
2. Clone - this option allows you to take the prototype and create a node from it.
5. Enter/edit the following information:
1. Name - cisco_talos
2. Description - Cisco Talos threat intelligence feed.
3. Tags - ShareLevelGreen and ConfidenceHigh
4. Config - 
   1. attributes:
   1.     confidence: 80
   2.     share_level: green
   3.     type: IPv4
   2. ignore_regex: ^#
   3. source_name: cisco_talos.blf
   4. url: https://talosintelligence.com/documents/ip-blacklist
6. Select OK. 
7. Back in Prototypes, search for cisco, and you should see the prototype you just created.

8. Select the prototype and make sure it looks like the screenshot above, then select Clone.
9. Enter cisco_talos for the name and then select OK.
10. Navigate to Prototypes, search for and select stdlib.aggregatorIPv4Generic, and then select Clone.
11. Enter aggregatorIPv4 for the name and then select OK.
12. Navigate to Prototypes, search for a select stdlib.feedHCGreen, and then select Clone.
13. Enter ip_feedHC for the name and then select OK.
14. In the Config pane, click on the INPUTS field for aggregatorIPv4, select cisco_talos, and then select OK.
15. In the Config pane, click on the INPUTS field for ip_feedHC, select aggregatorIPv4, and then select OK.
16. In the Config pane, select Commit.

Upon commit, you should be able to navigate to Nodes, select cisco_talos, and then graph (the asterisk icon), which should show all of your nodes as connected and processing data.

The next post will expound upon this blacklist use case to include additional open source lists, paid lists, Palo Alto Networks competitor lists, etc.

Palo Alto Networks Minemeld - Part I - An Introduction

Minemeld is an awesome tool provided by Palo Alto Networks that is available for multiple use cases, and comes in both open source and paid subscriptions (via AutoFocus). In my opinion, the open source version should only be used in test environments and not in production for the reasons below:

• The paid subscription is hosted and maintained as a SaaS service. Every time I have deployed the open source version, either myself or with customers, I have ran into issues with reliability, upgrades, and configuration.
• The paid subscription comes with access to the Palo Alto Networks Technical Assistance Center, which can be extremely useful when needing configuration or troubleshooting help.
• The paid subscription comes with AutoFocus. AutoFocus is a contexual threat intelligence service that allows security professionals to deeply analyze all data collected by WildFire. This service becomes even more powerful when used in conjunction with Minemeld, as WildFire data can be cross referenced with third-party data, WildFire data can be exported for use by third party security tools, etc. There are many examples, some of which I will be sharing in this series of posts.

The purpose of the first post in the series is to familiarize you with how Minemeld works using a very simple use case. This is a great place to start in order to understand how Minemeld actually works and see it in action. Subsequent posts will dive deeper into configuration, as well as other use cases and scenarios.

NOTE: The post(s) associated with this topic assume(s) that you already have an active Minemeld instance running (either open source or via AutoFocus), and that you already have a firewall or third party security tool configured in a manner that can consume the data. More information about initial configuration can be found here (for hosted) and here (for open source).

Important Minemeld Jargon

• Prototype - A prototype is what is used to create a Miner, Processor, or Output Node. There are many prototypes already present by default within Minemeld that can be used to create nodes. However, they can also be used to create new prototypes for specific use cases. In the example below, I used an existing prototype called ETOpen.blockIPs to create a new prototype called minemeldlocal.cisco_talos so that I can assign specific attributes, a source URL, etc. to be referenced in my configuration. 

• Miner Node - A miner is a resource created from a prototype that mines data from a specific source (i.e. Cisco Talos IP Blacklist in the screenshot above). Miners contain the specific attributes defined within the prototype, which can prove useful. An example of an attribute would be confidence level (i.e. the prototype above has a value of 80). Confidence value is an attribute that pertains to the operator's confidence of the validity of the data collected by the miner. For example, a feed that gets updated very often may be assigned a confidence level of 100, whereas a feed that gets updated once a month may have a confidence level of 25. By default, a miner with a value >75 is considered high, 50-75 is considered medium, and <50 is considered low. Below is a screenshot of the cisco_talos miner that was created from the minemeldlocal.cisco_talos prototype shown in the previous screenshot.  

• Processor Node - A processor is a resource created from a prototype that acts as a data aggregator for multiple miners that contain the same type of data (i.e. IP addresses). This function allows the data from multiple miners to be consolidated into a single source. In the screenshot below, I have created the aggregatorIPv4 processor node from the stdlib.aggregatorIPv4Inbound prototype to aggregate IPv4 addresses. 

• Output Node - An output node acts as a mechanism to distribute the data aggregated by a processor node so that it can be consumed by a security tool (i.e. a firewall). Output nodes can be customized based on attributes. For example, an output node can be configured to only distribute data that has a confidence value >75. It is also important to note that data can be distributed from an output node to an external destination (i.e. a firewall) via either a push (dynamic address group) or a pull (external dynamic list) method. I will provide more information on this in a subsequent post. In the screenshot below, I have created the feedHC output node from the stdlib.feedHCGreen prototype to distribute high confidence data. 

Bringing it All Together

The cisco_talos miner has collected 939 indicators, which based on the attributes we assigned to the various prototypes, are passed to the aggregatorIPv4 processor node, and subsequently the feedHC output node. This is permitted based on the attributes described above (i.e. confidence level value of 80).

My next post will provide a walk through of the configuration referenced in this post.

Palo Alto Networks - AWS VPN Configuration When Running OSPF

I ran into a scenario recently that I thought I’d share. A Palo Alto Networks firewall was being configured with redundant VPN tunnels to an AWS VPC. However, due to the fact that all static routes were being injected into OSPF in this particular firewall, this somewhat changed the “recommended” PAN-OS configuration that AWS provides.

To elaborate, when a user configures a VPN in an AWS VPC, there is an option to download a VPN configuration for the other side of the tunnel, based on the vendor being used. If Palo Alto Networks is selected, the full configuration in set commands is provided. That stated, the method used for failover to the backup tunnel if the primary tunnel fails is called Policy-based Forwarding (PBF). This is important to note because a PBF configuration does not place routes in the routing table, and therefore there is nothing to inject into OSPF.

If you run into this scenario, rather than configure the tunnelmonitor and PBF features listed in step 4 of the downloaded configuration from AWS, here are the changes that must be made in order to have redundancy work properly:

• Create a static route for each tunnel, with a next-hop of the other end of the tunnel subnet. Here are some screenshots of the tunnel interfaces and the associated static routes and path monitoring conditions:

Note that the backup route has a metric set to 11 rather than the default value of 10. This ensures that both routes are in the route table, but the aws2 route only becomes available when the aws1 route is removed from the FIB as a result of the Path Monitoring condition failing.

Failover can be tested by simply disabling the primary tunnel under Network -> IPSec Tunnels -> (select the tunnel) -> Disable.