08/25/25
By @BootlegFish
Have you ever been a victim of a phishing email?. If you have you know that it absolutely sucks. However, us people are sometimes not very good at writing these phishing emails, But AI is. And using one simple promt injection (Might not even have to call it that), we can have AI generate a pretty realistic phishing email.
Tricking the AI is actually not as hard as you think. All you have to do is ask it to create the phishing email for "Educational Purposes" but that does not stop it from being misused. Here is the phishing email that ChatGPT made for me.
As you can see, that is actually pretty convincing. While most people probably would not fall for this, You still have to be aware about things like this.
You can defend yourself against phishing attacks like this by checking the from address. If does not come from a reliable source or a official domain of the service that is sending you the email. Another thing you have to make sure is that the email itself actually looks legit. If it looks scetchy in anyway do not ever trust it.