Awareness training

[io-waiter]

Hi hivemind,

I’ve been looking into security awareness training for techsavy users and have so far narrowed interesting vendors down to NINJIO, Hoxhunt and Sosafe.

Does anyone have insight, experience and knowledge to share about this or potential competitors?

Any curated experience is much appreciated

TIA

 

[tstm]

I happen to work at Hoxhunt (or rather, I've started there about a week ago) but I've been using their product for several years in my previous jobs and I've been very happy with the quality of their product. The gamification works well, and seems to keep people engaged and on their toes, really hammers in that you should not be clicking on random links on emails or SMS etc. Part of the reason I agreed to join this company.

I really don't know much about our competitors, but I can tell that this product works well. I'm of course somewhat biased, but I've yet to drink at least all of the Kool Aid while here.

If you have any questions about the offering however, I'm sure I can find out and post here. =)

 

[zyyn]

Avoid KnowBe4. They try so hard to make their videos entertaining and engaging that it’s often hard to tell what the actual message is. I’ve also noticed that they spread misinformation such as the idea that forcing users to change passwords every couple of months increases security. Or the one about special characters being important.

 

[Paladin]

Since you brought it up, I wonder how many companies use a service like that to weed out the deeply stupid employees that just fail over and over?

I've known a few over the years that would just click anything in front of them no matter how much training you gave them.

 

[KD5MDK]

They also weed out the ones who insist on never clicking anything, like a button on the training.

 

[wobblytickle]

my experience is that there are n C-level types that just don't do them

 

[io-waiter]

Having looked at a several different vendors, most are meh and there is very little differentiation in content and engagement method. That said som look cool for phishing, HoxHunt and Adaptive Security stand out, but phishing might also be very much worth its own stand alone product. So far my users like the Ninjio cartoons, but I suspect something like Hoxhunt would be perfect after a year.

Some is also almost funny, we don't use real vendors logos or product names in our phishing, due to copyright, well that is certainly something all cyber criminals ...

 

[andygoblins]

The longer I'm in this industry, the more I become cynical of all security awareness training. It all sucks.

It's either outdated (like password advice) or it shifts blame onto employees and away from engineers.

Should it really be a user's fault if they click on a phishing email? Maybe IT should stop giving everyone admin access so end users can't accidentally install trojans.

The best training I've seen just tells people who they should call if things go wrong. Ideally, it's makes people feel safe, and they won't get in trouble for calling.

(Side thought: what if we just stopped giving email to all employees? Most communication has moved to other platforms anyway)

 

[sryan2k1]

Most communication has moved to other platforms anywa

Not even remotely true. Email remains the primary communication method for most organizations.

 

[andygoblins]

Ok "most" is a generalization. But it goes both ways; it's not accurate to say most comms have moved to other platforms, nor is it accurate to say it's the primary comm method for most orgs.

I am aware of many business units that don't communicate over email at all. Those units would be better off not having email and then not needing blame-shifty phish training.

The point is, I think there are creative ways to handle security without devolving into "training" that's just victim blaming. It's not the user's fault that security sucks.

 

[sryan2k1]

nor is it accurate to say it's the primary comm method for most orgs.

Yes it is.

I am aware of many business units that don't communicate over email at all.

Pay statements, HR, lots of reasons they still need and use email.

 

[andygoblins]

Payroll is a web portal. HR communicates through chat channel announcements. HR and Tech support use a unified ticketing platform. Corporate communication happens over zoom, with announcement posts on an internal web portal.

Just because you've never seen it doesn't mean it doesn't happen.

 

[sryan2k1]

I didn't say it doesn't happen, I'm saying you're grossly under estimating how much nearly every business is completely dependant on email.

 

[wobblytickle]

I fear that you @andygoblins and @sryan2k1 are somewhat talking at crossed purposes but sryan2k1 is absolutely not wrong with the importance of email; internally perhaps less so, externally, lord yes.

The point is, I think there are creative ways to handle security without devolving into "training" that's just victim blaming. It's not the user's fault that security sucks.

There aren't many creative ways to 'handle security'. Just to be clear you mean security training?

 

[andygoblins]

There aren't many creative ways to 'handle security'. Just to be clear you mean security training?

Security is fundamentally creative. Hackers, good and bad, reimagining how technology can be used or abused. The only rule is that the rules exist to be bent, broken, and reformed.

Security training deserves to be creative too, but I'm more concerned that the training is focused on the wrong thing. It usually focuses on outdated security tips and about shaming users with messages like, "you are responsible for your company's security"

As Bruce Schneier would say: training amateurs to take care of security gives you amateur security.

Do you know who is responsible for your company's security? The CISO and the security engineers. It's in their job title.

If your job title is Director of Marketing or Junior Analyst then it's not your job. If, for example, you write down your obscenely long password that changes every 90 days, you shouldn't be shamed for this. That's the security team's fault. They missed the 10-year-old industry guidance and Human Factors studies showing that these requirements aren't effective.

Don't get me wrong: I think there should be space to teach "street smarts" for using computers and the Internet, but it's gotta be done without the shame and blame. And I'm not sure this kind of training needs to go to all employees equally and become a bar we use to measure quality.

So TLDR; if the attackers get in and do something bad, that's not a regular employee's fault. Any training that tells them it's their fault is trash.

 

[sryan2k1]

If, for example, you write down your obscenely long password that changes every 90 days, you shouldn't be shamed for this. That's the security team's fault. They missed the 10-year-old industry guidance and Human Factors studies showing that these requirements aren't effective.

Real world in both a giant publicly traded company and now a private firm I work for

Us: Hey can we get rid of password expiration? Here's the NIST guidelines
The Business: Our contracts with (lists our largest customers totaling in some cases half a billion dollars a year) mandates them, sorry, they stay.

"you are responsible for your company's security"

Just like security is an onion, responsibility is shared as well. Not fucking typing your username and password into a site that is an obvious scam is something that is up to the user.

So TLDR; if the attackers get in and do something bad, that's not a regular employee's fault. Any training that tells them it's their fault is trash.

Have you never worked in the real world? You could have the best security in the world and Dave that gets a call from "The IT Department" will happily give their user and pass and MFA codes to someone on the phone.

 

[andygoblins]

Have you never worked in the real world?

Ad hominem. Refusal to believe that other's life experiences could be different.

I'm done.

 

[w00key]

Ad hominem. Refusal to believe that other's life experiences could be different.

I'm done.

Questioning why you think like this is valid critique. I have idiot users who get phished once in a while too and no matter what MFA and security rules you have, all you need is username, password and OTP to get a webmail session from which you can reset a ton of other credentials.


The most outrageous case was Uber support getting scammed by restaurant-name@gmail.com to

1. Add a new Uber Eats merchant admin user (what the flying fuck).
2. Ignore our shared instruction to verify a secret PIN for sensitive information after a previous, exactly the same attempt, account take by sending a mail to support@.
3. Ignore that there are several username@restaurant.nl users that have access to this account, so why the fuck do you do 1
4. Tried to change banking details, to divert tens of thousands of € per week.
5. Higher up said no, requires verification, and then they called the number on file and contacted us by phone. Result: blocked payouts until we send them a ton of paperwork proving ownership. Wtf? We are a merchant for many years now, how is #1 a thing that causes paperwork for us

We grilled that account manager when the weekend is over, but officially, there is no closure to this story. Is it an inside job? How is security that lax?


It's not always the end user getting scammed, the other way, scamming the help desk, works great too if you outsourced it to far east and they just do things without thinking.


You are right that the red team is ever changing, but the solution, the "system" or green team should be the only one responsible, is wrong. You need everyone to be aware, if spam filter malfunctions for a day, the second line - humans behind the desk, have to be on alert and not do their best to click everything. There is no 100% technical solution, if there was, we wouldn't have this discussion.

 

[andygoblins]

Ok it's morning again. Can think better. Still not convinced that

Have you never worked in the real world?

is a valid way to ask for clarification, so I'm gonna ignore that one.
I do want to spark on interesting discussion though, and want to explain my viewpoint:

1. Human Error is almost always bad design.
Bad design happens when we build something that works against the way the human brain processes information. If people are making errors, it means engineers need to come up with a way to prevent the errors.

Citations: A good popular book on this subject is The Design of Everyday Things by Don Norman. It's a fun read; I highly recommend.

2. Email is plagued with layers of bad design

My point isn't just email, but phish training is a common feature to training platforms so it's worth discussing.

• Senders can put whatever they want in the "From" field.
• SMTP servers accept all messages by default.
• SMTP validation methods are all add-ons to an already outdated protocol. They have poor adoption rates, which makes them even less effective.
• Email clients (especially Outlook' but also Gmail) are top tier examples of bad UI design. They go out of their way to hide the inauthentic markers of a phish. If you want to check a suspicious email, you have to click and hover on things to get important information.

3. Users aren't idiots
This is the hardest lesson I've had to learn. It's much easier to go through life assuming other people are stupid or incompetent. But reality is more complex and people deserve more credit than we give them.

I'm not discounting your experience in your last post @w00key. There are definitely Bad Actors. Maybe they have vendetta against the company. Maybe they're being paid by a Nation State. Or maybe, they are sleep deprived because their kid is in the hospital. Maybe they are in a domestic violence situation at home which really screws with their sense of worth and their ability to trust their gut.

That's a bunch of "maybes". I don't know your story and don't want to assume anything so instead of tearing into your story, let me tell you some of mine:

1. An employee was tricked by a corporate phish email that took the guise of an HR notice about canceled vacation time. The phishing failure went on his permanent record. The employee had only been at the company from a few months. He had just put in a request for vacation and was expecting an email. His previous employer did really shady things with vacation time so he was pre-disposed to click.

2. A Tech agent reset a password over the phone to a pentester. Technically followed policy, but the situation was sus and they should have known better. Got berated by upper management. But for context, Tech agents were naturally trained to err on the side of granting access if they wanted to keep their job. Management told agents to prioritize access. This directive came from C-suite. Also, the identity verification checklist was documented in a hard-to-access Lotus Notes database (seriously) and employees security question validation required looking up data in a mainframe. It was very onerous and error prone.

3. A development team pulled shenanigans to acquire software without the required security review. They were on a tight deadline issued by C-suite and the software solved a critical problem. The security review team had no people skills. They would openly mock/taunt people and intentionally delay processes to assert dominance. They also said "no" by default to every new software purchase.

4. If your security is actually good, you shouldn't care if a regular user clicks a link or downloads unapproved software

Good security design forces people to do the right thing my default.

Take away admin access so people can't install crap on their corporate devices. Enforce Least Privilege so the average Marketing employees can't access critical systems.

Don't let Windows Admins or Linux root log in remotely. Now tech support doesn't need to reset those passwords over the phone.

Run a proper EDR. Use a SIEM and have a good incident response team. Train your staff to have compassion for employees and focus on problems instead of attacking people.

Train your critical employees first. Empower them with the tools to get their jobs done with minimal inconvenience. Write short and concise security policies.

Then, and only then, can we talk about company wide training. But don't treat it as a checkbox. If employees are still making critical mistakes, it's probably not because of the training. Nobody sat through a 15 minute cartoon and walked away as a cyber security expert.

Blame the engineering. Blame the attackers who take advantage of people. Blame the execs who don't want to pay for quality security. But don't blame the users.

 

[w00key]

Train your critical employees first. Empower them with the tools to get their jobs done with minimal inconvenience. Write short and concise security policies.

Then, and only then, can we talk about company wide training. But don't treat it as a checkbox. If employees are still making critical mistakes, it's probably not because of the training. Nobody sat through a 15 minute cartoon and walked away as a cyber security expert.

But why A then B? It's not like you can have the green team deal with any and every eventuality.

Human errors happen.

There shouldn't be a blame culture, the first account hijack - eh shit happens. You have an established procedure for secret PIN? Let's use it then.

Second time around though, what the actual fuck? How are you giving out owner access like candy?


And this is both a system issue - account recovery isn't a new thing, both Google and Microsoft has a good procedure for it, but also a training issue, bla@gmail.com SCREAMS scam when every other user is username@bla.com.


My opinion is that it is a shared responsibility.

Your issues with training seems to be more about the shitty ineffective training forced onto every user. In addition, when basics (in step A) is not done yet.

My opinion is that yes, fix A first, but also make B less ineffective and a waste of time and just a on the audit form. Everyone needs to be on alert, when "Uber Eats" calls and wants your 2FA code, say no, let me (the boss) know immediately so I can send out an alert on the staff channels. I even received a reminder not to let "Uber Eats support" swap out your tablet, apparently this is a thing, they just enter your premises with a spare tablet and take away your logged in one, a very physical version of session hijacking.

We don't even have cybersecurity related requirements, basic training is just common sense and a good way to prevent getting owned.

We also gave a 1 minute refresher one someone got cryptod. Nbd, restore files on dropbox and nuke the machine, and hey everyone, try not to doubleclick any .pdf.exe okay.

 

[andygoblins]

But why A then B? It's not like you can have the green team deal with any and every eventuality.

My goal isn't to turn this into a false dichotomy (I can see how my previous wording sounds that way. Sorry about that). What I'm trying to say is that companies I've worked for seem to want a one-size-fits-all security training platform and then blast it out to all levels of the company equally, instead of taking a more risk-based approach that prioritizes certain people for more frequent quality training, and the pre-recorded training videos from various platforms like KnowBe4 just don't cut it.

Human errors happen.

There shouldn't be a blame culture, the first account hijack - eh shit happens. You have an established procedure for secret PIN? Let's use it then.

Second time around though, what the actual fuck? How are you giving out owner access like candy?

My point is "human error" is not a useful shorthand. Errors are complex failures affected by design, engineering, and social structure.

We all want that ideal blame-free culture, but then a user fails a second time and we quickly resort to "stupid support agent didn't follow the rules again".

We have to move away from simplistic "human error" then we start making real progress:

• The company has an account reset feature. Why didn't the agent follow it?
• Is the procedure easy to read and understand? Language is hard. Look at this forum thread! It's so easy to jump to conclusions about what a person knows or understands or feels
• Was that 15-minute training video with cute graphics and multiple-choice questions an effective way to communicate critical information?
• Was it clear to the agent that they MUST follow the procedure? Or is it possible it was interpreted as optional? Different cultures have different interpretations of authority, duty, honor, and shame that impact how they interpret their job requirements.
• What are the agent's incentives? Do they have to meet a call quota? A happy customer quota?
• If the agent follows through on the reset activities, they'll have to say a hard "no" to the caller. Will their manager have their back? Or will they be left on their own without support?

And this is both a system issue - account recovery isn't a new thing, both Google and Microsoft has a good procedure for it, but also a training issue, bla@gmail.com SCREAMS scam when every other user is username@bla.com.

But does it scream? I agree it's obvious to me but I can bit-bang the SMTP protocol from memory. I'm not exactly normal. Just because something is obvious to you or me doesn't make it obvious to everyone.

Human Factors studies and Cognitive Psychology tell us that the human brain is particularly bad at attention tasks that require picking out the one bad thing among all the good things. Sometimes people who are good at this can become impaired and lose their ability due to outside stresses - family struggles, health issues, etc. We're not wired to be hyper vigilant. It's not how humans work.

My opinion is that it is a shared responsibility.

Your issues with training seems to be more about the shitty ineffective training forced onto every user. In addition, when basics (in step A) is not done yet.

We're on the same page here. Despite my rhetoric, I'm not advocating to throw out training. But I do have a lot of concerns about the idea of shared responsibility, because I've seen it used as a weapon to shift blame from the people who can actually make a difference, and as an excuse to not spend more money on proper security controls and personnel.

I think that training, especially training that you buy from some company and then roll out to all employees and monitor using dashboards -- is ineffective, and propagates bad ideas, like Human Error. You don't use training as a blame-shifting device, and that's great! But I've been part of large corporations who did.

 

[w00key]

But does it scream? I agree it's obvious to me but I can bit-bang the SMTP protocol from memory. I'm not exactly normal. Just because something is obvious to you or me doesn't make it obvious to everyone.

If you're on tech support and have the keys to every tenant I expect basic proficiency.

Uber, a multi billion dollar company, has a decent self service, team management site. If they took a look at the tenant they would see that 1. There is an owner already. 2. The staff account is actively in use, like right now, accepting orders. 3. The email addresses associated with them.

This isn't a chef / line cook not being able to handle the control panel, this is god tier admin blindly adding owners to tenants.

If we disagree on this minimum skill requirement, to follow basic rules and not react to a "hi giev access please" email, then we have to agree to disagree.

We all want that ideal blame-free culture, but then a user fails a second time and we quickly resort to "stupid support agent didn't follow the rules again".

The second time was agent not following a big red flag on file on our profile. Require a PIN before doing anything, because you know, we got owned by their call center already.

And that is a systemic issue yes, but again, I expect better from god tier admins, most reply guys don't have that level of access to their systems. This isn't front line staff helping drivers or ride hailers resetting their account, this is about multi store tenants bringing a million+ in annual revenue, with its own 24/7 incl weekend and holidays hotline and assigned account managers, people we sat down with and had a meal together.

We didn't chase the account manager down to get the agent in trouble, we did so to figure out wtf happened and how to prevent a third time.

But I do have a lot of concerns about the idea of shared responsibility, because I've seen it used as a weapon to shift blame from the people who can actually make a difference, and as an excuse to not spend more money on proper security controls and personnel.

If people bend shared responsibility into it's always the user's fault, it's that company's culture problem, not the idea of shared responsibility.


In the case of a .pdf.exe, there isn't a scanner that catches everything and we only notice when it does fail. What's the next step, just lock out everyone to everything and whitelist applications? That's a rather Draconian rule and one that causes everyone a lot more work. Of we keep it practical, "hey, you can do anything except sudo/admin access, and be careful alright."

This pragmatic option depends on shared responsibility. On our own incidents, the user is apologetic, know shit happened and called for help. We fix it, don't give him shit and remind everyone else what to watch out for. The one who was pwned doesn't need the reminder, he certainly will be more careful from now on.

What's the alternative? User goes, eh, shouldn't have allowed me to do this? Full lockdown kiosk mode, browser only by default for all staff?

We actually did consider deploying Linux but it's still not mature enough yet, and if there's trouble, it is much harder to find others with the same issue just because of the install base. And we regularly need to use MS Office, especially the office with finance, hr staff and managers. So for now, the pragmatic choice wins.


Some issues you can fix with tech. If you rely too much on tech alone, you get different issues - Google's security is good, and you can do almost everything yourself, but find yourself on the wrong side, locked out, and it's good luck getting in touch with a human.

 

[andygoblins]

If you're on tech support and have the keys to every tenant I expect basic proficiency.

Uber, a multi billion dollar company, has a decent self service, team management site. If they took a look at the tenant they would see that 1. There is an owner already. 2. The staff account is actively in use, like right now, accepting orders. 3. The email addresses associated with them.

This isn't a chef / line cook not being able to handle the control panel, this is god tier admin blindly adding owners to tenants.

If we disagree on this minimum skill requirement, to follow basic rules and not react to a "hi giev access please" email, then we have to agree to disagree.

We can agree to disagree. I don't know your situation. But I can see you're working in Tech, which has access to different worker pools and expectations than the non-Tech industries I've worked in.

And that's the thing about security being an Art - you have to be creative and adapt to your own situation.

If people bend shared responsibility into it's always the user's fault, it's that company's culture problem, not the idea of shared responsibility.

I agree.

In the case of a .pdf.exe, there isn't a scanner that catches everything and we only notice when it does fail. What's the next step, just lock out everyone to everything and whitelist applications?
...
What's the alternative? User goes, eh, shouldn't have allowed me to do this? Full lockdown kiosk mode, browser only by default for all staff?

Unfortunately, this is the next step. Application allowlisting is a common requirement in compliance frameworks. It's not quite as bad as you think--you can create expansive lists of allowed software and let people self-serve install as needed, plus more liberal rules for engineers.

You don't have to like it (I don't), but it's important to know that there are options.

------
At the end of the day, I think we're talking about different concerns. You are concerned about a skilled technical worker pool (upper-tier support staff, engineers, developers, etc). I agree these people have shared responsibility for security. It's probably in their job description.

My main concern is a non-technical worker pool (logistics managers, marketing agents, tax specialists). It's not their fault that security is hard. We engineers made the world hard for them. I take this as my responsibility. I need to engineer things better, and constantly question how "easy" I perceive something. Because my definition of easy is not inclusive.

 

[io-waiter]

Allow listing is extremely good and combined with hardening it forces the antagonist to go to whatever LOL tools they find or face the eye of Sauron… implemented right it is not a barrier for usage. Also it makes security a technology issue instead of bogging down people with things they are not played for.

As a side note Ninjio looks nice and I will update more later.