I call this meeting to order.
Welcome to meeting number 25 of the House of Commons Standing Committee on Access to Information, Privacy and Ethics.
Pursuant to Standing Order 108(3)(h) and the motion adopted by the committee on Monday, December 13, 2021, the committee is resuming its study on the use and impact of facial recognition technology.
I'd like to welcome our witness. We have today, as an individual, Nestor Maslej, research associate at the institute for human-centred artificial intelligence, Stanford University; and from the Privacy and Access Council of Canada, Sharon Polsky, president.
Mr. Maslej, you have up to five minutes for your opening statement.
Good afternoon. I'd like to begin by thanking the chair and members of the committee for the invitation to speak today.
I'm Nestor Maslej, and currently I serve as a research associate for the Stanford Institute for Human-Centered AI. I am also a co-author and the lead researcher for the AI Index. Although my testimony today makes use of data from the AI Index, I am speaking as a private citizen, and my views are not representative of those of the Stanford Institute for Human-Centered AI.
The AI Index is an annual report, currently in its fifth edition, that aims to track, distill and visualize key trends in artificial intelligence. Our goal at the index is to be the best and most authoritative single source of information on trends in AI. The index aims to give policy-makers like you not only a deeper understanding of AI but also, crucially, an understanding that is grounded in empirical data.
It is this latter aim especially that informs my testimony today. I am here to answer the following question: What does data tell us about facial recognition technology? I will answer this question by tackling two sub-questions. First I will comment on capability. As of today, what can FRT do? Second I will examine usage. Who uses FRT—public and private actors—and how?
In terms of capability, there has been tremendous progress in the performance of facial recognition algorithms in the last five years. The index looked at data from the National Institute of Standards in Technology's face recognition vendor test, which comes from the U.S. Department of Commerce and measures how well FRT performs on a variety of homeland security and law enforcement tasks, such as facial recognition across photojournalism images, identification of child trafficking victims, deduplication of passports and cross-verification of visa images.
In 2017, some of the top-performing facial recognition algorithms had error rates anywhere from roughly 20% to 50% on certain FRVT datasets. As of 2021, none has posted an error rate greater than 3%, with the top-performing models registering an error rate of 0.1%, meaning that for every one thousand faces, these models correctly identify 999.
The index also shows that the performance of FRT deteriorates on masked faces but not by an overly significant degree. More specifically, performance is five to 16 percentage points worse depending on the FRT algorithm and dataset.
In terms of usage, FRTs are becoming increasingly deployed in both public and private settings. In 2021, 18 of 24 U.S. government agencies used these technologies: 16 departments for digital access or cybersecurity, six for creating leads in criminal investigations, and five for physical security. Moreover, 10 departments noted that they hoped to broaden its use. These figures are admittedly U.S.-centric, but they paint a picture of how widely governments use these tools and towards what end.
Since 2017, there has also been a total of $7.5 billion U.S. invested globally in funding start-ups dedicated to facial recognition. However, only $1.6 million of that investment has gone towards Canadian FRT start-ups. In the same time period, the amount invested in FRT technologies has increased 105%, which suggests that business interest in FRT is also growing. Our estimates also show that FRT is the 12th-most funded area out of 25 AI focus areas.
Lastly, a McKinsey survey of leading business executives, which we include in the index, shows that across all surveyed industries, only 11% of businesses had embedded facial recognition technology in their standard business processes, which trailed robotic process automation at 26% and natural speech understanding at 14% as the most embedded technologies.
In conclusion, I've presented some of the AI Index's key findings on the current capabilities and usage of FRT. It is my hope that the data I have shared usefully informs the committee's deliberation on the future regulation of facial recognition technologies in Canada. I'd be more than happy to answer any questions on the data I've presented and the implications that it may have.
Thank you so much, Chair, and good afternoon, members of the committee as well. Thank you for inviting me to appear before you today on behalf of the Privacy and Access Council of Canada.
My remarks today reflect round tables held by the council with members from across the public and private sectors, and with members of law enforcement, who agree that facial recognition is one of many digital tools that have great potential.
Like any technology, facial recognition is neither good nor bad, but it's easy to justify, especially when considered on its own. What people do with technology makes all the difference in reasonableness, proportionality and impact on lives.
Thirty-four years ago, our Supreme Court said that “privacy is at the heart of liberty in a modern state”, that “privacy is essential for the well-being of the individual” and that privacy “is worthy of constitutional protection”, and I dare say it still is, except that now we struggle to have any privacy, at home or away.
It's difficult now, if not impossible, to prevent our facial images being captured and analyzed and our movements and our associations being calculated and evaluated in real time. We are in view every time we go outside, and often inside as well, and our images are posted to the Internet, often without our knowledge. We haven't consented to those images being used, or to our gait, our keystrokes or other biometrics being analyzed and correlated with databases that have been amassed with information about each of us.
We haven't asked that the voice-activated devices or the messaging platforms that our children use at school and we use at work analyze our conversations or our emotions, or for our TVs to watch us watching them, yet that is now commonplace, thanks to governments and companies creating an unregulated global biometrics industry that's predicted to reach $59 billion U.S. by 2025, while the tech companies embedded in the public sector urge us to use our faces to pay for groceries and to get government services.
In the 40 years that computers have been part of our daily lives, though, there hasn't been any substantive education in Canada about privacy or access laws, or rights or responsibilities, so it's no surprise that Canadians trust that the laws themselves are enough to protect privacy or that just 14% rate their own knowledge of their privacy rights as “very good”. In the meantime, there's been an onslaught of automated, privacy-invasive technologies and multi-million dollar investments in surveillance technologies to create safe communities across Canada purchased by the other 86% of people as well.
Certainly, facial recognition-enabled cameras in cars, body cams, doorbells and cellphones might help police identify a suspect or solve a crime, but even police admit that cameras and facial rec do not prevent crime, and there's little correlation between the number of public CCTV cameras and crime or safety, yet their unregulated sale and use are a self-fulfilling prophesy, because familiarity breeds consent.
Facebook, Cambridge Analytica, Cadillac Fairview and Tim Hortons are just the tip of the iceberg. Companies and governments can and do create or use technologies that violate our privacy laws because they can, because the current consent model is a fantasy, and because Mark Zuckerberg and others know that the risk of penalty is far less than the reward of collecting, manipulating and monetizing information about us.
We are at a moment, though, where three important changes are needed to help safeguard our democratic freedoms without impeding innovation and so that Canadians can regain trust in government, police and the public sector.
First, enshrine privacy as a fundamental human right for all Canadians, in person, online and at our borders.
Second, enact laws that require everyone who creates, purchases or uses technology to demonstrate that they actually have a clear and correct grasp of our privacy laws, rights and responsibilities.
Third, in the same way that vehicles and food must meet stringent government regulations before being allowed for sale or use in Canada, craft laws that put the onus on creators, requiring that technologies undergo comprehensive independent examination of their privacy access and algorithmic integrity, bias and impact before the product or platform may be acquired or used, directly or indirectly, and make sure the standards are set and the laws are written without the direct or indirect influence or input of industry.
Those are just a few highlights of a very complex issue that I am looking forward to discussing with you.
A perfect example is that you'll typically see the introductory fluff, “We respect your privacy,” and then it's, “We will collect your personal information only for business purposes,” and a list of other vague terms. Every for-profit organization's business purpose is to improve their bottom line and their net profit. Anything they can do to fulfill that obligation is a legitimate business purpose, as far as they're concerned. It's meaningless when it comes to protecting individuals. When we say yes to any of these, the companies essentially have carte blanche to share our information with their business partners, whoever they might be, wherever in the world. When it's outside of Canada, those companies do what they wish with it, for as long as they wish.
My apologies for that trouble with the microphone.
The AI Index doesn't comment directly on the amount of investment that exists in that space, but it alludes to the fact that this is becoming an area of increasing concern across a lot of different spaces. I'll highlight two points here.
The first is that with the data I provided from the NIST FRVT test, this test looks at 1:1 verification. In a brief that I submitted to the committee, in figure 1.1, I show the success of different models on a variety of different datasets. Now, in this figure, one of the things that is very clear is that the best-performing model is the one that performs on visa photos, and that's the one where you had a correct identification 999 times out of 1,000, whereas the worst model is the one that is for the wild photos dataset.
The wild photos dataset is a dataset of individuals whose faces might be obscured partially by shadows, or perhaps they weren't looking directly into the camera, and the top-performing models identified correctly 970 out of 1,000 faces. It's still very high, but there's a noticeable drop-off compared with the visa photos.
I think this is suggestive of the fact that if companies and agencies want to use these technologies and justify them on the grounds that, “Look, we tested them in the lab and they had a very high accuracy rate in the lab,” there has to be an attempt to qualify the difference between the settings in which these technologies are tested and the settings in which they are deployed. I think the committee is aware of this, but the index suggests that it is a pressing concern.
I will also add that we cite some research that was published a couple of years ago—and that I think has appeared in the committee as well—in the form of a 2018 paper by Timnit Gebru et al., entitled “Gender Shades”, which looks at the fact that a lot of facial analysis benchmarks tend to be overwhelmingly composed of light-skinned individuals, leading to subsequent bias, and that existing algorithmic systems tend to disproportionately misclassify darker-skinned females as the most misclassified group.
We allude to this, and I think there is a general sense in the research community that there should have been more work being done in this space, but I would be unable to comment as to the exact amount of investment that is being put into this particular field at the moment.
It might be outside of the scope of my area of expertise to identify ways in which they could be improved.
I would, however, say that in the paper you're citing, the issue they talk about there is “domain shift”, which is the fact that very often the settings in which some of these algorithms are tested are radically different from the settings in which they are deployed.
At the minimum, there ought to be some kind of clarity and honesty in terms of the agencies that use these tools—whether it's companies or agencies—about the extent to which a difference exists between testing conditions and conditions in which these tools are actually deployed. I think it would be problematic if there were such a big discrepancy, that these tools were tested in one setting but then deployed in completely different settings. If there isn't a clear sense and a clear understanding as to whether this difference exists, then these technologies perhaps have a great likelihood of being misused and serving more nefarious purposes.
Again, I didn't contribute to that report directly, so I wouldn't be best suited to answer the question, but I think perhaps the issue that the report is getting at there is one of institutional shift.
You might potentially have these technologies used by different individuals in different parameters. Certainly, being trained in the usage of these systems can be important, but I think there's also a recognition here that unless there is some kind of set regulatory standard about what is an acceptable benchmark or an acceptable framework, you might have different jurisdictions using these technologies in different ways. On the question of what this acceptable benchmark is, again, that is outside my area of expertise. I will leave it to you policy-makers to crack that one.
I think the point being made is that if a threshold doesn't exist, it's a lot likelier that individual agencies will make these assertions themselves. There are reasons certain agencies might favour lower or higher thresholds, and this can lead to potential misuse with some of the technology.
I thank the witnesses for joining us.
Ms. Polsky, if we define understanding as the ability to grasp everything that is at stake, I conclude from your statement that there is a great deal of misunderstanding among people, governments and users—in short, everyone who is involved to a greater or lesser degree in facial recognition.
I have three questions for you, Ms. Polsky.
In reality, the consent we give by clicking on “I accept” is not a choice. We have no choice but to consent. Is that right?
There were many.... Not only am I the president of the Privacy and Access Council of Canada, but for roughly 30 years I've been doing privacy impact assessments and privacy consulting privately. Through that, I have been invited inside everything, from governments and public bodies to Fortune 100s.
I understand how they operate, the technologies they build and what they deploy. There are some that sincerely believe they're doing it right, but remember, without education, they're misled. They're maybe assessing their own understanding a little favourably.
You get into situations where it's a Canadian company and they store the information in Canada, but they use third parties in the United States to provide essential services for that app or that service. Companies don't recognize that as a problem. They don't notify users. They don't have any concern. They're ignorant, totally unaware that there is a privacy implication.
Certainly. That's an excellent question. It speaks to the fact that although regulation of facial recognition technologies certainly matters, there are also other AI use cases that might merit regulatory attention.
One of the takeaways from the data we have on facial recognition is that facial recognition is in the middle of the pack in terms of total private investment. It's more invested than things like drones or legal tech, but it's behind NLP in medical and health care, which suggests, as well, that there are other AI use cases that might merit further attention.
As the data from the McKinsey survey suggests, facial recognition is not as embedded in certain business technologies and processes as are other AI systems. Again, this doesn't necessarily mean that we shouldn't care about facial recognition regulation; it's an issue of utmost importance. It's just that we should also be cognizant of the other problems AI might pose, especially at a time when AI is becoming increasingly ubiquitous.
You alluded to a couple of the different examples that we cited in the report. I'll speak to a couple of different ones.
We talked about the fact that there are résumé-screening systems that have been shown to discriminate based on gender. We cite evidence from a newspaper article that a couple of years ago, Amazon developed a machine-learning résumé-screening system that was shown to systematically downgrade the applications of women.
Again, it would be ideal for a lot of these companies, especially the very big ones, if someone gave them 100 résumés and they could just give them to a machine that says automatically that these are the best three candidates and just hire these three people.
The reason Amazon trained a biased system was that the system was ultimately trained on data from résumés that were submitted to Amazon previously. Overwhelmingly, the résumés that tended to be submitted to Amazon were submitted by men. This reflects the fact that the tech industry is mostly dominated by men. Given that men were mostly traditionally hired, the AI system learned to penalize the term “women”. That meant that if you included a resume that, for example, said, “I was captain of the women's swim team,” the algorithm saw that historically very few women have been hired at Amazon, this person has “women” in their résumé, so let's downgrade this résumé. Amazon claimed that this tool was not actually ever deployed to make hiring decisions, but the point stands that this bias remains.
We also talk about bias in multimodal linguistic models. We talk, as well, about bias in medical image segmentation. I could go on at length about that, but I'll perhaps give you the opportunity to pose additional questions.
Perhaps not necessarily as a best practice but a point of reality, one of the big takeaways from this “AI Index” report is that AI is becoming increasingly ubiquitous in all of our lives.
Ten years ago, there were a lot of AI problems that were very difficult to solve. This meant that AI was something that was just being researched, whereas, if you move forward 10 years, AI is now one of those things that are coming out of the lab and moving into the real world. A lot of companies are very excited about using AI technologies, and you're going to start seeing them used more and more. Investment in AI is going through the roof, and the number of AI patents is going through the roof.
Very often, I would say, a lot of companies are quite keen to use AI before perhaps coming to terms with some of the negative ways in which it can be deployed. As a regulator, very often it might be worth asking, when should we care about this? When is the time to regulate? I would say that—
I have to cut you off. I'm very sorry to do so, but we're getting further behind.
I'm going to have to cut the times for the subsequent rounds, and I still think we might end up having to squeeze a little past 5:30 to get in a few minutes of committee business.
We're going to go with four minutes each for Mr. Bezan and Mr. Fergus, two each for Mr. Villemure and Mr. Green, and then four each for Mr. Kurek and Ms. Hepfner.
Go ahead, Mr. Bezan, for four minutes.
AI is obviously something that changes day by day. I mean, 2022 has been a tremendous year of AI progress; it seems like every week there's a new model that's breaking ground. I don't think we're ever going to get to a point where we'll have data to sufficiently know the answer to every question, but we're getting more data, and an absence of absolute data does not mean that we shouldn't take action.
We know, for instance, as I stated earlier, that a lot of these facial recognition systems perform a lot worse on these kinds of wild photos, photos where individuals are not looking into the camera straight, or where lighting is not super good, and that might have important implications for how these technologies are regulated.
We're still far away from getting to a point where we're going to have data to answer every single question, but we are getting more data, and I think the data we have at the moment is sufficient to take action on certain different issues.
I'm not an academic. I leave that to you, sir, but I go back to news reports out of the Welsh police, where the senior-ranking officer said that facial recognition—and I'm paraphrasing—came up with something like 92% false positives, and he said that was okay, because no technology is perfect. That was in 2017. In 2020 or 2021, the chief of police of Chicago, I believe it was, said that facial recognition was something like 95% erroneous.
That can have profound implications on people's lives, because once you are identified as a person of interest, you're in the system, and then any time a cop looks at you, they run your name and you're already there. There's already a presumption that they should look a little more closely at you, because the facial recognition got it wrong.
Thank you, Mr. Chair, and I'd like to thank the witnesses for being here today.
Through you, Mr. Chair, I would have asked this question of both our witnesses, just following up on what Mr. Green asked, but given that our other witness didn't want to pronounce on this issue, I will ask this question of Ms. Polsky.
Ms. Polsky, in answering a question from my colleague, you indicated the problem of false positives and the extremely high percentage of false positives, on the order of 19 times out of 20. Given that you have pointed out those numbers, and given that a number of our witnesses before this committee have pointed out that we should place a moratorium on the use of facial recognition technology by the public and perhaps even by the private sector until a framework for this technology is put in place, do you feel that there should be a moratorium? Would you agree with those witnesses that there should be a moratorium on the use of FRT in public spaces as well as private spaces?
I'm sorry, Mr. Maslej. Let me interrupt you there. I have very little time—
The Chair: You have none.
Hon. Greg Fergus: Oh, drat. Well, this will be very quick.
You pointed out that there's a reason to give some greater consideration to the use of this technology, but wouldn't that lead you or the report to come to the conclusion that there should be a moratorium until greater certainty or greater accuracy can be brought to bear for the use of this technology? Can you answer yes or no, if possible?
The Chair: It will have to be yes or no. We're out of time.
Part of the reason they're becoming more biased is that, typically, these models are being fed increasingly large numbers of data. For certain models, it is advantageous to have a lot of data. The more data you give the model, the more likely it is to get some data that is not ideal.
We saw this in the report with this model clip, which is a multimodal linguistic model. This model was asked to assign the probability of an American astronaut, Eileen Collins, being.... The model was asked, “What is this image of?” The model assigned a higher probability that this photograph was of a smiling housewife in an orange jumpsuit with the American flag than that it was of an astronaut with the American flag.
That's not our finding. That's a finding from a paper of Birhane et al., 2021. It's illustrative of the fact that when you give these data a lot of models, which might be required for higher performance, they might catch some conspiratorial and biased data. If we're not filtering that data proactively, it could be very likely that these models behave in toxic and problematic ways.
I want to talk about the second recommendation. I appreciate that it's very helpful, just as a note for witnesses, for recommendations to come forward, because that certainly helps committees in the structure of the reports we are able to put together.
On the second recommendation, when it comes to the broad, society-wide database, how could such a database cause harm and possibly result in abuse of those who may be found in that database? The inverse of that is, for those who are not in that database, how could they possibly be harmed as well?
It's interesting, because recently I was watching a television show, and that was one of the case studies with profound possible impacts.
I'd like to move from the tenancy example, if I could. There's been a collection of data at Toronto Pearson, the international airport that I'm sure all of us on this call have been through countless times. It includes, I can only imagine, an unbelievable amount of data about Canadians, individuals visiting our country, and everyone in between. I'm curious if there are any further thoughts, recommendations, or concerns that you would highlight, and how an organization like an airport, or another entity, with law enforcement—
Yes, that's a great question.
I would say that the reason you have seen this technological advancement is that AI systems are getting better across the board. There have been a lot of developments in the architecture that powers these systems. These systems now run on better hardware, which means they're able to operate at much faster speeds. There are a lot of academic and business reasons that these systems are operating better.
In terms of consequences, again, all of this portends and points to the fact that AI is going to become a part of our lives whether we like it or not and, as I have said to you today, there is a lot that AI systems can do, but there's also a lot they can do that we didn't expect them to do or didn't perhaps want them to do. Rather than just welcoming these systems into our lives with open arms, it is important to ask what kinds of effects they might ultimately have.
Again, if we live in a world where, for instance, FRTs are now having incredibly high success rates, it might be a lot easier for companies to justify that they ought to be used, but again, that doesn't necessarily imply that we shouldn't think critically about how they ought to perhaps be regulated or managed by government officials.
I think it will be important to mandate that for the Privacy Commissioner—and this goes for the individual provinces as well that have substantially similar legislation—all legislation require that, first, the privacy commissioners' offices be fully funded and that they have a separate fund, also fully funded, for education. They don't all have an education mandate. They do some work to educate, but there needs to be a much more formalized program, because that will translate into people being more aware of the legislation and their rights and responsibilities.
They can build that into the technology, and then, once they have the technology, one thing that could be done is to test it in a sandbox, a neutral sandbox run by the Privacy Commissioner, as an opportunity not only for the commissioners and civil society groups to examine it, but for the corporations to allow it to be examined in a trusted neutral setting that does not violate their copyright or their intellectual property. That way, it gets tested and approved before it's allowed for sale in Canada.
Also, fund the education through the Privacy Commissioner's office, and again, without the influence of industry, please.