If you're a parent, you should probably be careful about the photos of your children that you upload and share via your Google accounts.
At least two parents in the U.S. learned this lesson the hard way after they lost access to their Google accounts over images that were mistakenly flagged by the tech giant's system for detecting child sexual abuse material (CSAM).
According to The New York Times, one of the parents, who is referred to only as Mark, took photos of his son's penis after he noticed it was swollen and hurting him. These photos were supposed to fulfil a pre-consultation request by his son's doctor. The other parent was in a similar situation, snapping shots of his son's intimate parts, which was believed to be infected, to send to his paediatrician.
Both parents reportedly used Android phones to take the photos, which were then automatically backed up in Google photos and sent to their spouses and consulting medical professionals using various services. Google's system then detected the photos, flagged them, deactivated the parents' accounts and reported the case to authorities.
Google's system isn't trained yet to detect the intent behind the photos or their context, so even in the absence of malice, there's always that chance they'll be flagged. Google product manager Hunter Walk said that it's better for the system to have false positives than false negatives. In the two parents' cases, the photos detected were false positives as the purpose was strictly medical.
Google did note though that its team of child safety experts consults with paediatricians to better identify instances where users may be seeking medical advice.
“Child sexual abuse material (CSAM) is abhorrent and we’re committed to preventing the spread of it on our platforms," said the tech giant. "We follow U.S. law in defining what constitutes CSAM and use a combination of hash matching technology and artificial intelligence to identify it and remove it from our platforms."
The two parents have since been cleared of any suspicions of abuse by investigators, but Google has yet to restore Mark's access to his account.
The report claims that Google's decision on Mark's account involves a video he previously uploaded "of a young child lying in bed with an unclothed woman". Mark, however, said that the video was a "private moment" that he didn't think would have been viewed by anyone else, adding that he no longer had access to it. A Google spokesperson said he could appeal the decision if he wanted to.
At least two parents in the U.S. lost access to their Google accounts over images that were mistakenly flagged by the tech giant's system for detecting child sexual abuse material (CSAM).
Both parents took photos of their children's intimate parts to send to their respective paediatricians for medical advice.
Google's system detected the photos because they were uploaded to Google Photos and flagged them.
The two parents have since been cleared of any wrongdoing by investigators.