This was about 10 years ago. I was working at a company that built applications used by state governments. This particular application was around getting your birth, death, and marriage certificates. Nothing super fancy, but was my first real dev project.
The app hadn’t been live for two weeks when I got a very urgent phone call from my manager on a Friday night. I was being called into a meeting with our VP of IT, a few devs, the admin of the records facility, the Attorney General and Governor of one of the states that used our app. I had an email with redacted emails and chat logs with the customer.
My stomach was in a knot as I read that a customer had gotten access to somebody else’s personal information. I was reading through the attached emails trying to figure out what had happened. Long story short, customer ordered a certificate and he received somebody else’s birth certificate as well as an email with all the personal information of this other person. I was in full panic mode. We had our QA team frantically trying to replicate this, Sr. devs going through the code trying to figure out where this could have happened. I was still reading through chat logs with the upset customer and checking audit history for when this order was placed. But the weird thing was I couldn’t find the rep who helped him in the system at all.
The customer kept referencing how he called and talked to Karen, but nobody was in our system named Karen. I had to stop digging and call into this meeting with all of these people.
As soon as I announced who I was and what I did for the application I was yelled at by the Governor who screamed “I want this developer fired, it’s all his fault that we had this happen!”
Luckily I had a decent manager who defended me and explained that I was needed to troubleshoot. So instead of getting fired I just got an earful for 20 minutes. Only then I was asked if there was anything else I needed to say.
Me: So the customer says Karen helped them place the order. Who is Karen?
Client: Yeah, I thought that was weird too, but I didn’t think too much of it. Are you going to fix your code or not?
Me: Hold up, somebody is accessing the system that you don’t know who? Did anybody ask the customer what website or phone number he called? I’m not sure he even used our system. Can I get the unredacted emails from the customer where the email addresses are still visible?
This resulted in a lot of silence until the governor finally said “we’ll get back to you.”
Once I got the unredacted emails, it was super easy to see what happened. Turns out the customer had gone through a third party that the Attorney General was trying to shut down for years. He went through them, they scammed him out of a few hundred bucks but ordered the certification through our system. When entering his information, autocomplete put in the other customer’s information on the order form. I was told that the Attorney General thanked me for helping them build a case. Never did get an apology from the Governor, though.