Tuesday, October 22, 2019

Palo Alto Networks Minemeld - Part III - Additional Miners

This post elaborates upon the previous previous posts in this series. If you haven't read through parts 1 and 2, I highly recommend that you start there prior to moving forward.

NOTE: The post(s) associated with this topic assume(s) that you already have an active Minemeld instance running (either open source or via AutoFocus), and that you already have a firewall or third party security tool configured in a manner that can consume the data. More information about initial configuration can be found here (for hosted) and here (for open source).

The purpose of this post is to build onto the previous configuration to further illustrate how Minemeld can be used to prevent threats using a wide variety of data sources. Your Minemeld instance should currently look like this (navigate to Config):


The additional data sources used below are just examples, but could be considered a good baseline with which to start. Feel free to choose others that may be more applicable to your specific deployment. Add the following miners to the configuration by navigating to Prototypes and selecting the Clone option:

  • alienvault.reputation
  • bambenekconsulting.c2_dommasterlist_high
  • bambenekconsulting.c2_dommasterlist
  • bambenekconsulting.c2_ipmasterlist_high
  • bambenekconsulting.c2_ipmasterlist
  • blocklist_de.all
  • dshield.block
  • ETOpen.blockIPs
  • feodotracker.badips
  • feodotracker.ipblocklist
  • itcertpa.DOMAINS
  • itcertpa.IP
  • itcertpa.URLS
  • libraesva.LIBRAESVA_Malware_Domains
  • phishme.Intelligence
  • proofpoint.EmergingThreatsDomains
  • proofpoint.EmergingThreatsIPs
  • ransomwaretracker.RW_DOMBL
  • ransomwaretracker.RW_IPBL
  • ransomwaretracker.RW_URLBL
  • recordedfuture.DomainRiskList
  • recordedfuture.IPRiskList
  • spamhaus.DROP
  • spamhaus.EDROP
  • usom.DOMAINs
  • usom.IPs
  • usom.URLs
  • zeustracker.badips
The Config will look like the following:


Add the following processors to the configuration by navigating to Prototypes and selecting the Clone option:
  • stdlib.aggregatorDomain
  • stdlib.aggregatorURL
The Config will look like the following:


Add the following output nodes to the configuration by navigating to Prototypes and selecting the Clone option (note that you will do this multiple times based on the output formats required):
  • stdlib.feedHCGreen
  • stdlib.feedMCGreen
The Config will look like the following:


For each processor node in the Config, add all relevant miners as inputs. The Config will look like the following:


For each output node in the config, add all relevant processor nodes as inputs. The Config will look like the following:


Upon commiting the Config, your Minemeld instance should look like this (navigate to Prototypes -> alienvault_reputation -> GRAPH):


As shown above, all miners are now sending data to their respective processor nodes, and based on confidence level value, being added to either a high or medium output node. Note, you will likely see a red exclamation mark next to some miners under the Nodes tab. This is because some of these miners require an API key, user name and password, etc. in order to gain access to the data. 

2 comments:

  1. It seems like most Palo Alto devices have a limit of 50,000 entries for EDLs but the Alienvault.Reputation prototype currently has over 87,000 entries. How do you handle that? (and thanks for the informative posts)

    ReplyDelete
    Replies
    1. Hi Inertia64, I just saw your question. As you stated, there are limits based on the different models. I typically try to be very prescriptive with my EDLs so that I don’t have a ton of stale entries. One thing you can do is age out anything older than a specific time period (i.e. 5 days). Another thing you can do is limit the number of total entries. Here is a thread I found that shows both options:

      https://live.paloaltonetworks.com/t5/MineMeld-Discussions/filter-based-on-age/td-p/274416

      Here is another option that would allow your to collapse contiguous IP addresses into a single object.

      https://github.com/PaloAltoNetworks/minemeld-core/issues/109

      This would greatly reduce the number of entries while still allowing you to potentially consume all the data.

      Hope this helps!

      Delete