Performing Due Diligence on a Security Assessment Vendor

As you begin the process of evaluating, hiring, and holding accountable, an IT Security Vendor, there are several key components you must evaluate prior to forming a partnership with that company.  Most of the article will apply regardless if the company you are interviewing is for the initial security assessment, the remediation, or the managed security service provider (MSSP or MSP).  I always recommend you have separate agreements for each phase.  And, if you decide to use the same vendor for multiple phases, I always recommend having a separate vendor for annual, or bi-annual audits of their work.

PART 1 – Evaluation and Hire

When reviewing security vendors, I recommend you evaluate a minimum of 2 – 3 vendors.
Security Vendor must understand the customer’s business.  Ask what they know about the business they are working with and examples of other clients in similar industries they have worked with.  Definitely ask for recent references.  Are they reputable?
Is security their only business, or do they practice in other areas?
Ask for a copy of their own internal policies.  How organized and documented are their processes and procedures?
Ask what are they going to do for your organization?  What types of tests will be performed?  What types of tests are available?

Security Assessments (identify symptoms and root causes)

Policy and Procedure Review / Gap Analysis
Privacy Review
Physical / Structural Review
Vulnerability Assessment / Penetration Testing
Host-Based Configuration Review
Application and Web Application Assessment / Penetration Testing
Network and Wireless Assessment / Penetration Testing
Security Program Strategy Review

Penetration Testing (Pen Test)

Network and Firewall Penetration Testing
Application and Web Application Penetration Testing
Physical and Structural Penetration Testing
Social Engineering
Wireless Penetration Testing

Privacy Programs

Privacy Program Gap Analysis

Regulatory Compliance

NIST Cybersecurity Framework
Financial Services Compliance
Sensitive Data Discovery

How intrusive is their process to the business?  What all is involved and who needs to be involved?  What am I responsible for?  What liabilities do they have?  Is there a clause in the contract that covers those liabilities?
What software / hardware do they use during the testing process?  (Tenable Network Security – Nessus, WhiteHat Security, RapidFire Tools)
How do they handle such things as BYOD and IoT during the assessment process?
What technologies are they familiar with?  Will this line up with the current technologies in use at the business?
What certifications does the business currently hold?  What about partnerships?
What certifications do the employees hold?  (Certified Information Systems Security Professional (CISSP) or Computer Hacking Forensic Investigator (CHFI) CISM, CISA, CRISC, etc.)
How long have the employees who will be performing the assessment been working in security?  How long with the company?   (Consider writing into the contract that final report must be reviewed and audited by someone with long-term experience)
How do employees stay current with new trends?
How long is the assessment process and what is typical turnaround time for producing a report with substantial content?
Ask for a sample of a report they have done for another company, redacted of course.
Do they provide just a list of current vulnerabilities, or do they go further by providing recommendations for remediation?
How much does an average assessment with a client of similar size typically cost?

PART 2 – Methodology

A simple scan will simply not do!

Requirement Study and Situation Analysis (Also called a GAP analysis)
Security policy creation and update

Including in Handbook? Is this part of the initial hire package?  Does the employee sign off on all requirements?  Is there routine training done?

Document Review
Risk Identification
Vulnerability Testing

Testing: Exercising one or more assessment objects to compare actual and expected behaviors, typically performed using automated tools

Identify systems, ports, services, and potential vulnerabilities
Techniques include: Network discovery, port and service identification, vulnerability scanning, wireless scanning, application security examination
Console Auditing; reviews internal systems, servers, and network equipment for configuration and potential security audits.

Target Vulnerability Validation Techniques

Performed Manually or with automated tools; Password cracking, Penetration testing, Social engineering, application security testing.
Web Application penetration testing
Physical (Building level) penetration testing.

Assessment: Documentation review, log review, ruleset and system configuration review, network sniffing, file integrity checking, etc.
Interviewing: Conducting discussions

Data Analysis

Examination: Checking, inspecting, reviewing, observing, studying, or analyzing assessment objects


PART 3 – The Good, The Bad, The Ugly

The right and wrong ways some security vendors perform their assessment

The Good:

Hands-on Technology Skills (must have hands on experience. Need to know the nuts and bolts of how it all works)
Ability to Embrace Complexity and Craft a Balanced Response
Reject Fear
Efficient Communication
Accepts Responsibility
Intelligent Interpretation
Fascination with People
Confidence without Arrogance


The Bad:

False Security – Convinces management of security state that does not exist
Misdirected Investments & Waste
Breaches / Attacks
Lack of Vision
Letting compliance get in the way – approximately 30% of compliance requirements are bad news
Bad Reporting – make sure to get a copy of their reports and be happy with the format.


PART 4 – What to look for in the final report and briefing

Introduction/background information
Executive and Management summary
Assessment scope and objectives
Assumptions and limitations
Methods and assessment tools used
Current environment or system description with network diagrams, if any
Security requirements
Summary of findings and recommendations
The general control review result
The vulnerability test results
Risk assessment results including identified assets, threats, vulnerabilities, impact and likelihood assessment, and the risk results analysis
Recommended safeguards


Do you feel like you got the value?