Introduction -
This post is part of our AI6P interview series on “AI, Software, and Wetware”. Our guests share their experiences with using AI, and how they feel about AI using their data and content.
This interview is available as an audio recording (embedded here in the post, and later in our AI6P external podcasts). This post includes the full, human-edited transcript.
Note: In this article series, “AI” means artificial intelligence and spans classical statistical methods, data analytics, machine learning, generative AI, and other non-generative AI. See this Glossary and “AI Fundamentals #01: What is Artificial Intelligence?” for reference.
Interview -
I’m delighted to welcome as my guest on “AI, Software, and Wetware” today. Devansh, thank you so much for joining me today! Please tell us about yourself, who you are, and what you do.
Hello, Karen. Thank you so much for having me. I'm an AI researcher and writer that covers a lot of important ideas in AI technology from a technical perspective, also how technology and AI and other developments will impact us as a society and its people, et cetera. I work with a software services group called SVAM, S V A M International. And I write on my newsletter called that's available for free on Substack.
And you're based in the US? You mentioned you worked with an international company?
Yes. The company, SVAM, is international. I am in the headquarters in New York. The company operates in 5 different countries, so that's why international.
Nice. Very good. So tell us about your level of experience with AI, machine learning, and analytics. If you've used it professionally - sounds like you have - or personally, or if you’ve studied the technology formally or informally?
Absolutely. So I am actually one of the builders. A lot of my work is actually how do you build AI solutions from the ground up. What happens? How do you work with different technologies?
I've been covering AI research since around 2017. My first foray into AI was Parkinson's disease detection on voice calls in real time in much, much older technologies and cell phones. So imagine that you have bad networks and old cell phones. Traditional machine learning at that stage wasn't really capable of handling those low-resource, high-noise environments. That's where I got started.
Since then, I've worked across multiple fields, multiple areas. AI has been something I've been doing professionally for many, many years now. And, obviously, since the start of ChatGPT, there's been a lot of interest, a lot lot more. It's become a lot more mainstream.
So since then, a lot of my focus has also shifted, not just towards building AI, but also, communicating with other people. How do you use AI for your use case? How do you build technology in a way that's useful to you, as opposed to something that might not be great for your business? There might be other technologies that are more applicable. So just helping people even assess whether or not their AI usage is valuable in the first place.
Yeah, that's a great perspective. One thing that I've always advocated is that you start with the simplest thing that could possibly work. A lot of solutions simply don't need AI to be solved well. Sometimes non-AI solutions can be better. And it's good to stop and think before just diving in and trying the latest shiny new tool.
Exactly. There are times where technology is great. There are times where technology as a whole might not be useful. For instance, with creators, you know, YouTube is very egregious with this. Bots that do copyright content reviews.
So often big channels or media companies will abuse the system to strike down independent creators. They'll flag copyright violations where there were none. And then people can get their livelihoods deleted. People can get, like, their channels permanently put on hold. That's one of the use cases where, like, do you want AI? Maybe you want a human in the loop. That's how you set it up that's ethical.
You know, even with AI, there's this big push towards large language models being deployed everywhere. But they're not necessarily the best for every use case. So how do you know where to use them and where you might be better off using another strain of technology? It's a very rich and deep and complex field, because no worthwhile problem is solved simply. And I think that's really my job is to identify where one solution might be more helpful than another.
So you mentioned that you build AI tools. I'm assuming you've also evaluated and that you use AI tools. Could you talk a little bit about that?
The tools we evaluated?
Yes.
So, obviously, we're evaluating AI tools by the hundreds. And I think you can largely classify and categorize them into multiple categories. And so, firstly, of course, we have the AI tools that enable the development of what we build. So that would be something like a large language model. That would be something, like, “Oh, we're going to pull this code out. We're going to process this. We're going to run this in some way.” So that's more of a platform or something that we build on top of. That's primarily what I spend most of my time evaluating, to see, “Okay, are these secure? Are these stable? Do these work well? Do these not work well?”
Those are also where you have most of the misunderstandings. Because, again, I think there's a big divide between what you think how these tools work and how they actually work in the background, or where you think they might be useful and where they might actually be not so useful.
For instance, Gemini’s Flash 2 model that came out. A lot of people are very, very impressed by it, and frankly so, it's a very cool model, and it's been breaking all these benchmark performances. But what we noticed is it's a very unstable model. So if you give it an unclear context, it actually can backfire its results. If you ask it to rank, “Hey, here are 3 or 4 different cases. Give me a ranking of each of them.” It might come up with similarity scores that are very similar because it gets influenced by previous contacts and previous chats. There are certain text inputs that can trigger an infinite loop, that it just generates one thing over and over again.
So these are all, like, concerns that we're always trying to monitor, which is: “Can I read beyond what the headlines are? Can we study beyond what's going on and studying them?” And this is where most people that write about them, they don't actually interact with the tools extensively. They'll only run the code once or twice on 1 or 2 things. Say, “Oh, this looks very good.” But as a builder, you have to be much, much more rigorous. That's primarily what I spent time evaluating.
Other than that, there are quality of life engagement tools. There are tools for content creators. Can we take videos and include them in VR/AR tools that can take my articles, convert them into podcast form, that I tried out a little bit. You know, I wasn't too impressed by them, because they made me sound more American than I am. Usually, it's supposed to be the other way around, but they've completely butchered my accent. And I was like, “Wait, that does not sound like me.” Anybody that listens to a podcast like this, and then speaks to me, is going to wonder what <laughs> what have I done? So that, unfortunately, did not work out so well for me.
So evaluating tools like, “Oh, can I create videos from articles? Can I use AI for outreach that'll give me a bunch of leads, and then reach out to people for me for my business development work?”, et cetera. Evaluating them to a lesser degree because that's not really my field of expertise. And there's just so much going on in the technical research area that it's not THE most important for me to know what's going on here, as opposed to the building-oriented tools. But I evaluate them more casually, content creator tools or whatnot.
Well, I promise we're not going to use AI-based tools to process the audio for this interview, so you WILL sound like you.
Thank you. Well, that is becoming a concern for me more and more.
Right. So you've talked a lot about what you do for your job. I'd like to hear a little bit about what you're doing in your personal life, and with your newsletter about AI, and if you're using any AI-based tools in conjunction with that activity.
So, one, I mentioned my job because writing is also part of my job. It's a large part. My writing acts as a way for me to share my research insights with people, and then they share their insights back with me, so I can get an understanding.
But specifically with writing, I think I will come out and say this. I don't like it when people use AI primarily as a tool to write. I think if you have to tell AI to tell you what to cover and then how to cover it, I think you shouldn't be talking in the first place.
There's this big push these days. “Oh, we're going to use AI for content creation.” Like, ChatGPT gives you video insights, then you use ChatGPT to write a transcript, and then you use another AI tool to come up with the video. I don't think it's a good way to use it. I'm not a huge fan of that.
But I do like the AI for writing, okay? A lot of the times, what I'm covering, I myself am not fully aware of it. There are ideas in cutting-edge technology that obviously I haven't come across. It's not as though I know everything there is to know about AI.
So I’m using something like ChatGPT or Gemini or Claude or any of these other technologies to ask them questions like, “Hey, this is a formula that I don't fully understand, why it's structured this way. Can you help me understand?” Often I'll come up with hypotheticals like, “Oh, I think because the formula has a square root here, or a square here, or they're using this particular way to process this data, I think that's what it'll do to it. Does this idea make sense?” And that will kind of act as a sanity check, almost.
Now I'll also often put these out in the writing itself and say, “This is my thoughts. What do you guys think about it?” But these things are almost like a database that I can ping my ideas out of to see, “Oh, does what I'm thinking make sense?” Or ”What is this work that they have done? How does this play a factor?” That has been very, very helpful to me.
These have been trained on so much data. They have so much different, I won't say knowledge in the way that we have it, but they definitely have so much information encoded into their systems that it can often be helpful to get their perspectives on things. And that's really how I've found it very, very helpful. So as the assistant to my thinking, for sure, I found them to be great.
Yeah. I've talked to a lot of writers and most of them say, as you do, that they don't want to use it for content writing. The whole point of writing is to have their own voice and to do their own thinking, and they don't want the tool doing that for them. But others will say that they use it for things like analyzing something they've written for continuity and pointing out where they've got a gap that they should fill. Or brainstorming titles that would be catchy.
I've used it sometimes, but not a huge concern.
You mentioned the voice and changing it. I looked into voice cloning tools last year. And I found some of them, a lot of them, were trained unethically. A few seem to be on the more ethically-sourced side.
Interesting. The voice cloning tools also? I know this is true for a lot of the image generators. I don't know for the voice cloning tools as well.
Quite a lot of them. They advertise, you know, “Do something in the voice of any celebrity that you want”, and they list all these celebrities. And you know perfectly well those celebrities did not consent to their voices being used for all of these purposes.
I'll have to check the name of it, but I did find one voice cloning tool [Speechify] that was using famous people, like Gwyneth Paltrow and Snoop Dogg. Those were actually authorized - they have a partnership with those specific celebrities. But it's a handful. It's not the famous politicians or the famous music stars.
That's interesting. Now that you mentioned it, I think it would be an interesting discussion to have with somebody who would be more insightful on these topics. But, how much of a celebrity's voice or likeness does actually belong to them? And who gets the rights for this?
Like, if I was to take a celebrity's picture and use that for a video thumbnail, you know, reasonably, you can say that I am using it, fair use of that thumbnail, and whatnot belongs to me. I know I can play songs while I'm streaming a video game or something, and in that case, I'm not violating their copyright. So would me building a voice cloning tool that takes what is publicly available information, such as an interview, and then processes it and creates a tool out of it, like, would that be considered fair use? Where does that end?
And who should I pay, if I'm taking? If somebody uses our conversation to create a voice cloning tool of me, should they pay you, as the person who owns the platform? Or should they pay me, as the person whose voice was used?
I think that would be an interesting discussion to have. Not sure that I have anything useful to say right now, but that's one of those areas of technology that I think people haven't really fleshed out too well at this stage.
Well, you're probably aware there are over 30 active lawsuits in the US on the rights to use music and voices and such, and art. And there was the ELVIS Act that was passed in Tennessee last year, basically protecting what they call the N I L rights - Name, Image, and Likeness. And the ELVIS act was specifically aimed at musicians and famous people. Question is: what about the rest of us? What about those of us who are not famous?
One of my friends in Costa Rica that I interviewed in December, she had a friend who is a voice artist. And someone else used a voice cloning tool to clone his voice. And now they're making money from his voice. And I think it's pretty obvious, everyone's going to say, “Well, that's not fair.”
But when it comes down to celebrity voices, there was the incident with Sky. The voice of Scarlett Johansson was used by OpenAI for one of their voices. She refused her permission for them to use it, and they used it anyway. And there was a big fuss about that last fall. So there's a lot of active discussion around voice cloning and what's ethical and what's not, and the way that the tools should be used.
The streaming services DO have to pay. There are systems in place. And music is interesting. The reason I started looking into it last year is that there's a more established tradition of regulating, and ensuring that artists and musicians and performers get royalties, and that their copyrights are protected. And so it feels like music is breaking some ground here that may apply to other aspects of the law around AI, and how people's content and their intellectual property and their rights to their Name, Image, and Likeness get used. It's a really interesting area.
Right. Yeah. I would love to see more work being done there, because you see this all the time with speeches. There'll be a lot of edits where somebody's using somebody else's voice in the background. A lot of the times, what people will do is to circumnavigate anything. They'll download the voice, and they barely make any edits, but they upload it, and now it's considered their original product. So just more conversations around this.
Maybe because I'm a creator myself, I have always been put off by how easy it is to reuse and repurpose people's content, and not give them any credit. Or, just generally, how much duplicated nonsense gets published online. I would love to see more work being done there.
Yeah, there was a recent study that reported that over 50 percent of the content on LinkedIn is now AI-generated. So there's a lot more of the slop or this noise content where someone AI-generates a post. And then someone feeds the post into an LLM, and gets back a reply, and then they post that. It's diminishing the signal among the noise in a lot of the conversations that are happening online.
There might be MORE original content with generative AI. Because one thing I noticed with LinkedIn before has been that, a lot of people, pre-ChatGPT, were just copy-pasting posts anyway. They would copy a big post, and then they’d just be, “Oh, credit to this guy.” Often, there wasn't even credit to this person.
Actually, one of the reasons my newsletter is fairly well-liked is the memes I make. And one of the reasons I decided to specifically go down the route of making my own memes was that every time I would see posts on social media, I'd be like, “I've seen this before. I've seen this exact meme in 5 different places before.” Like, 3 different LinkedIn accounts. 2 different Facebook groups. Somebody shared on Instagram. I just saw so much content repurposing going on there anyway that I thought, “Hey, at least let me make sure that the memes people see here are original.”
My point being that, with LinkedIn and I think with ChatGPT and whatnot, maybe that's because there's been such a pressure for generative AI creative posts and comments. And because these systems themselves are not deterministic, there's going to be some variance in them. You might actually see more content diversity, like, large-scale mainstream adoption of those, as opposed to prior, where a lot of the times, the post which is straight-up recycled.
But, again, I don't think either of those are a good sign. I think both of those are fundamentally very problematic, both societally and technically, for many reasons.
Yeah. That's one good thing that I've noticed with some of the AI-based tools. They have tools that generate content, but they also have tools that can check for plagiarism. I'd love to see that run on some of the AI content that we see on LinkedIn. Or even some of the non-AI content where people, like you said, are just copying and pasting, basically stealing someone's full posts outright and not giving them credit.
I know the plagiarism tools have had some issues in schools, where they are falsely marking students' content as plagiarized, and it's having a lot of consequences for them. So the false positives can definitely create problems there. But it's interesting that we have tools developing on both sides of this. It's a bit of an arms race.
It's quite an interesting journey. I think I have fewer comments there, I’ll say, but there are a lot of interesting areas where I would like this to be explored further.
Right. Well, that was a really fun detour, but we should probably get back to your questions!
It sounds like you've done quite a lot with AI and ML tools and different features. Can you share a specific story on a way that you've used such a tool, and how well the AI features worked? What went well, what didn't go so well?
Sure. So one of the projects I can talk about publicly, working with a company called IQIDIS, some legal AI startup. And there, we've been building up a lot of solutions, to help speed up lawyers' workflows. How do you make lawyers more efficient? And I think that is where a lot of both the good and the bad of these models, these different AI tools, can come very well into demonstration.
The goal is to speed up. There's a lot of inefficiencies in the way lawyers work right now. So how do you speed this up? This involves stuff like, okay, document review can take a long time, especially if the document is long. So at IQIDIS, we built out a tool that can process documents much, much, much more quickly. And there are lots of ways you can do this.
A lot of the standard solutions don't work for lawyers. Because often, with a lot of legal documents, you might have handwritten notes, you might have annotations. You might have information in charts and tables and whatnot that doesn't get transcribed as well from a standard PDF process. So they just don't work well. This includes a lot of top cutting-edge solutions. They often miss the mark on many of these points.
So, how do you build on there? That's something that we worked on with IQIDIS. Got some decent results, but also, there's a lot of work to be done there. Just making sure that all abbreviations got correctly. Different states will have different ways of approaching and building up a case argument. So how do you make sure the formatting is correct? How do you make sure the arguments are correct?
That has been one of those areas where you've been able to see some places we're able to speed up workflows tremendously, because we can help you build up a case argument. “Oh, these are my points. Can you help me retrieve the right cases, the right bills that might support this or might counter this?” These places, AI actually works very, very phenomenally.
Other places, such as “Oh, this kind of a document should have 4 spaces here, exactly 3 columns here” - doing that, they’ve struggled very, very hard with the precision-level work. And that's been something that we're always actively looking to improve and build on.
So I think, generally speaking, AI is getting very, very good at doing 60, 70, 80 percent of a lot of the work. And then the last 20% is one of those very detail-oriented, very, very specific jobs that:
1, you need a lot of domain expertise for,
2, you need a lot of precision and trial and error for
That's been very hard to control and very hard to build on. That is where, currently, I think the next generation of solutions would be. Can I just give it a few comments and points, and can I have it follow those guidelines perfectly without any deviation? And can I do so cheaply and efficiently?
Because the current ways for doing that are not sustainable, either environmentally, commercially, or just computationally. So that, I think, has been a great area for future exploration that many people are working on actively, not just ourselves.
Yeah. The area of the law and applying AI for it is really interesting. There were 2 big concerns that I've heard come up.
One has to do with the confidentiality of the content that a legal firm might put into the tool, and whether or not their client's confidentiality is protected from the provider of the tool, or the platform behind the tool.
And the other concern that has come up is the accuracy of the citations, or what some people call the “hallu-citations” [credit to Charlotte Tarrant of ], when a tool makes up a citation that doesn't exist, and needing to review 100% of them, because you wouldn't trust a legal assistant or a paralegal to make up citations. It’s something that really can't be tolerated. So how do you work with that, or is that an area that you're tackling?
You mentioned legal citations, and I think that's a great example. We have zero error with legal citations. That's what I meant earlier, where people misunderstand what these technologies can and cannot do well. So the large language models, you don't want to trust them to do very, very precise work like citing something. We have a legal database that we query and we are able to retrieve. So it's more like, “Oh, for this case, this is what was mentioned”.
The issue happens when you try to use technology. You can’t ChatGPT or whatnot for legal citations, because the way they're set up, just algorithmically, there's always going to be a rate of error there. And with most use cases, most technologies, that's acceptable. I doubt you care if you're using ChatGPT, for example, to generate a memo, whether it uses the word can or the word could. Does it use the word “They can.” “They might be able to do this.” “They will be able to do this.” “They have the potential to do this.” Like, these are small errors that are generally acceptable.
But if you're starting to use ChatGPT for generation … for example, we work with a biotech firm. And one of the things there is the drug C1103 and the drug C1104, that's an error bit of 1. But out of, like, if it's, like, a 100-bit code for it, and you have one bit that's wrong, that's still a 99% accuracy. Anywhere else, that's very, very good. With this, you flip one character, you've created a completely new different compound chemical with a very different chemical structure!
So you cannot afford to be using unreliable, undeterministic technology there. You want something that works 100% of the time as it should. Or when it doesn't work, you know where it doesn't work, so you can work around.
So that is where you need to have both very strong domain expertise and very strong technical understanding of how these products work, to ensure that you use the right solution for the right places.
You asked how we solve the legal hallucinations. For our citations, we're pulling from a legal database. So any citation we give you, we're giving you from a legally verified database. On top of which, we're also saying, “Hey, this web item, you might want to double-check this.” So you only have to double-check the citation, not the whole argument or the whole work.
If we're doing document review, one very simple solution there is, if somebody gives you a document and says, “Hey, what is the relationship between this party and this party?” Or “What does this party say about this statement?” You can give both the analysis, and also, this is the relevant area in your document that they're finding this information. So suddenly, instead of you having to read through the whole document to verify this one answer, you have a direct source to your point.
So, you know, the same way academic research works with citations, right? I say, “This is the one part I'm citing from this paper”, so that anybody who wants to double-check my work, who wants to build upon our work can just go see the paper, see, “Okay, this is the part on that paper where the citation is useful.”
If you just follow those basic first principles, you've suddenly dropped the amount of extra work and errors you have from 50, 60, 70% to 10%, 5%, which is where the other part comes in. There's a lot of talk about ‘AI solutions replace people’. “Oh, I'm going to replace my whole developer team with this code programming bot. I'm going to replace my whole sales team with a customer support bot.” Please do not do that. That's incredibly stupid. What you can do is, now that I have a 5% error rate, 10% error rate, that's very acceptable, because now I have a human in the loop who can check this work.
And humans now just have to check the work, make sure all the things are good. And suddenly, the productivity goes up, even if you have those errors. Because where before, it would have taken you 1 hour to do the work, now it's taking you 5 minutes to generate the answer, and 15 minutes of verification. So that's 20 minutes. So there's a 3X differential in productivity, even if before you did 15 minutes of drafting and 10 minutes of checking. So your checking time has gone up 50%, but your overall work time was down significantly.
So that's the kind of work you do. Once you understand who your user is, you understand how they operate and work with your specific solution. Then you can build solutions that either don't have those errors or, if they have the errors, the delta between having those errors, checking those errors, but doing everything else that can really speed up your work, is good enough to where you got some of the acceptable options. That's generally how I think I can view technology as well.
Thanks. So we've talked a lot about ways that you've used and worked with AI-based tools. Are there any situations where you have avoided using AI-based tools for some things? And if so, can you share an example of when and why you chose not to use AI?
Image generation is a big one for me. One, I don't use something like DALL-E or Gemini. Until they start crediting the people whose work they're using, they start paying them, and, like, they openly say, like, this is fully 100 ethically sourced, I would stay away from them.
Also, just from a computational perspective, those are very expensive to generate images. People don't understand how much of the computing load they can put on your computing budget. So that's where I like to avoid using them.
Similarly for search, there's a lot of people who use ChatGPT almost as a search engine, or Perplexity as a search engine. I don't like that. So similarly, these solutions cost a lot more than a simple Google search or a Bing search or whatnot. So where I can, I'd like to use these technologies just to keep the environmental footprint down so that the loads in our solutions are lower.
As mentioned earlier, I don't like to use them extensively for writing everything. I think if people are reading my news, I have a duty to deliver to them what I have to say or what other people have to say.
I might use them for drafting specific pieces. Again, as I mentioned, I often use them to understand certain explanations, to look at certain ideas, see if that makes sense. So they have been a helpful tool in my writing. But my writing is mostly me. And I think that's something that becomes very, very obvious when you read it, that it's very human-generated.
So those are just some of the areas where I like to use them. Professionally, I think I tend to avoid AI in, again, anywhere where I can't control the outcomes, and there's too much of a risk.
So I don't like AI for evaluating employees, for example, potential employees. People use ChatGPT for evaluating resumes, and that can have so many unintended consequences. For instance, I actually ran this experiment myself. I changed the name on my resume and the email. Nothing else. No location changes. No other changes. Just the name and resume email, to, like, different races, different genders just to see what that will be. And I ran this a bunch of times, and I'm like “Hey ChatGPT, please score this out to 100.”
And across the board, like, I think, when I give it my own name and when I give it, like, a white person's name, both for men and women, there was like a 40-point difference out of 500. It's not a massive, 5 tries, run this, sum them up. But there's, like, a 45 point difference there, which is close to a 10% scoring difference. And if you're getting a lot of resumes, that's a fairly big shift.
And now where that becomes a problem is, sometimes it gave me a very good review on the resume. Other times, it actually had a much, much lower score for me than other people. Even though identical resumes. Literally identical resumes. It's just my name, with the names changed.
So that kind of stuff is where I get a little bit concerned. I'm not saying they will necessarily discriminate. In some rounds, it gave me a very good score. What I'm saying is there's a chance it discriminates. And there's no way for you to know when it's discriminating and when it's not discriminating.
So that's where I would avoid using such a technology if possible. Or if you're using such a technology, then maybe do something like a personal information reduction, so that you're limiting the sources of bias that come in. You're running each evaluation 5, 10, 15 times so that the variance of that goes down a little bit. Those are all just some of the ways that I think you want to be very, very careful with technology, because I believe that can hurt people more than it can help.
We started this conversation while talking about YouTube's bots and broken copyright system. That's one huge area where - I mean, I have no personal power over this, but - I would flag this very, very strongly, if you're using AI to review content made by people. That to me seems like it has a whole bunch of red flags.
So you've made some really good points about avoiding AI for image generation and being very cautious about anything that involves human resources and potential biases. I know those are both huge areas, and you're certainly in very good company. A lot of people have raised those concerns, especially about the AI-generated images. And a lot of people have talked about the “3Cs”, the Consent and Credit and Compensation [credit to CIPRI], that creative people whose content gets used are entitled to.
And I'd love to hear your thoughts. There's a bit of an arms race of image poisoning tools, like Nightshade and Glaze that are used for protecting images of artists' work, and then the AI companies are taking countermeasures toward that. Is this something that you've looked into at all or had any experience with?
Aggressively. I think I'm probably one of the only major AI research people that talks extensively on “adversarial perturbation”, which is the technology that tools like Nightshade, et cetera, are built on.
One of the big reasons why we spend so much time evaluating these tools is you can have both active and completely accidental attacks that shut these systems out. With regards to Nightshade, et cetera, there are problems with them right now, which is that:
1, they're not very reproducible, and
2, those are often very architecture-specific,
which is why they haven't gotten mainstream adoption. But one of the people I'm working with is interested in scaling these adversarial perturbations out. And we're going to be looking at how you build out attacks that work across different models, that can really corrupt these models. So that basically these AI companies have no choice but to start sourcing data more ethically.
Yeah. I think a lot of people are hoping that regulation will come along and make everybody do things ethically, but regulation always lags. I think the more that we can do, either:
as consumers putting pressure on companies that we won't buy or use tools that aren't ethical, or
looking at the technological solutions to counteract it and kind of force them to work with ethically-sourced images, as you're talking about with the adversarial perturbation.
That's a really good initiative, and I'm happy to see that. I enjoy what you're writing about all that.
See, I'm a little bit skeptical on regulations, just because the people who make regulations are politicians, and you can lobby them. There's more lobbying for it. There's more lobbying power for the AI tools, as opposed to against this stuff. So that's where I'm a little bit skeptical on how useful or impactful the regulation will be.
And that's where I think, like, sometimes you just have to do things yourself until there's enough of a power imbalance. Or you can address that power imbalance, and you can come to the negotiating table as an equal.
Because if you're relying on regulation to solve problems for you, I feel like in my mind, you're also just saying, “Oh, I'm not capable of solving this myself.” And I think that's a very dangerous mentality to have.
Yeah. I think a lot of people feel kind of powerless about the use of their data and about tools that aren't operating ethically. But I think we have more power than we realize, and we shouldn't just give it up without trying.
Exactly. I think that's one of the few trends that I don't like. I'm generally very laissez-faire about technology and social media, but I think that's one of the trends that's always been concerning to me is there seems to be a trend towards powerlessness and learned helplessness and saying, “Oh, let's wait for politicians or whatnot to fix this - my own politicians doing more to align with my principles.” As opposed to, “Oh, what can I do to truly have power and influence change in the way that I would like it?”
And that's probably why I've gravitated so strongly to adversarial perturbation. I'm very concerned about surveillance technology, the use of AI and tools and whatnot, tools used in surveillance. I can start tracking and start potentially coming after you, send that information to your employers, to be like, “Hey, these are the people who are protesting inside.”
This is where I think we need to actively work. We need to fight against the business structures and incentives to truly take back our power because nobody else will. Regulations, politicians, Silicon Valley, Elon Musk - whoever else you like, worship, love, whatever - they're not going to give you your power. You have to take it yourself.
Yeah. That's a really good point. And I'm happy to hear that there are people fighting back. It's good to see more of that happening nowadays.
There's been a lot of news stories lately about discovering that companies are using data that we put into online systems or that we published online. Or in the most recent case with Apple, where they're surreptitiously recording conversations and then selling them to advertisers and without consent, even from people that thought they weren't using Siri at all. So there's a lot of cases where there's a sense that companies really can't be trusted to do the right thing, and we have to find a way to make them do the right thing.
Oh, hang on. I did not know this. Apple has been selling information to advertisers?
I'll send you the link. The story broke on January 2nd that Apple was settling a lawsuit. For, I think, 10 years starting from 2014 or so, in iOS, they were using Siri to record conversations. And then they were selling the information from the conversations to advertisers so they could market to us, even people that thought they had Siri turned off.
That is insane. The whole picture about Apple being the privacy-protecting software just goes right out of the window.
Yes. I mean, they talk a good story. And we all thought they were trying fairly hard to do the right things. And then this kind of story comes out.
And I was having a conversation with someone earlier today about this. It's like Adobe with Firefly. You know, they say that they only trained on good images, and it looked really good when they first announced this last February. But then it turns out that they've used some Midjourney images to train the tools. And they also, in a lot of cases, didn't compensate people who have used Adobe for years and hosted their images on Adobe servers, and those artists didn't get compensated.
So the question then is - if it's an honest mistake, that's one thing. This Apple thing sounds like it pretty much had to be deliberate. That infrastructure, it doesn't come into place. Yeah, that doesn't sound like an accident. But then what's going to be interesting, I think, is to see what Apple does to recover and make up for that. And I think the few millions of dollars in the lawsuit settlement doesn't even come close to what it needs to be. That's not enough. They need to do something to earn back our trust.
Yeah. That is a wow. Another reason not to use Apple, I guess.
Yeah. I want to talk a little bit more about consent and credit and compensation. We mentioned that earlier. So it sounds like you feel pretty strongly about it for images and forcing companies to do it right. In general, just talk a little bit more about what you think about consent and compensation, especially?
I mean, it's kind of a generic question. Those are obviously extremely important. The reason I feel much more strongly now than I did in the past about that is also because it isn't difficult to build the technology required to provide this to some degree.
For instance, let's say I generate an image. I'm like, ”Hey, give me a video, or give me an image of Vegeta from Dragon Ball bench-pressing with a koala.” Now, it's not hard for me to say, “Okay, I generate an image”, and then I'm just like, “Okay. In my training data slate in space, here are 5 images that look very similar to this.” Or maybe ”There's no bench-pressing with a koala anywhere. But here are 5 training data samples that are, based on this vec, this is the representation of this data point. Here are the 5 closest vector representations in my database.”
The problem, I think, is you can't mathematically prove whether one training data point was used to influence, and how much did that influence, et cetera. So that's probably why you don't build it naturally. But even if you show, like, “These are the 5 closest points, check them out.” I think that provides some level of credit. You can do a top 100. It can be like, “Okay, search through these.”
Another fairly easy fix you could do is something like, “Oh, I'm asking you for a …” Most images are really not asking me something that's never been conceptualized before. Most things are like, “Oh, give me a picture of, like, a dog playing with a ball” or something. Not super duper difficult. What if you were to first show human-generated images that exist in this training dataset? To be like, “Hey, here are some data training points that we have. These are good. And if none of these fit you right, then sure, we'll generate something for you AI-wise.” That'd be good enough.
That way, you not only first give preference to the people who have built your AI solutions, but also what this does is, it's much more environmentally efficient. Because suddenly you're not running the much more expensive image generating tool until after you have to create something.
So those are just some fairly simple solutions that most people could build that would not be that expensive. They would not increase their operating costs too much. And they would still provide a lot of benefit.
And, in any of these structures, you notice, none of these actually involve paying anybody. Just like the same way, if I quote your article on one of my articles, I just have to credit you. I don't have to pay you money for it.
This follows that very similar value structure that I think does work fairly well for the most bit, and it's not perfect, yes. I can always make a statement like, “my training dataset also included Vegeta that I contributed, but I didn't get credited for it.” And you can have problems there. But I think that's still so much better than what we have right now, which is nothing. Which is kind of where I feel strongly, is that an imperfect solution for crediting people is still miles, miles, miles better than no solution for crediting people.
Obviously, you can also just pay people to build. Like, I'm buying the rights to use your data, and then I'm not going to credit you. And if you were doing that, there are AI solution tools like Nvidia AI. That was one of them that I've played around with. I asked them. They said there's no crediting of any sorts in that image generation. But they did say, “Listen, we're using the datasets. It's commercially licensed, it’s ethical.” I was like, cool, I have no problems with that. Because if you're paying to buy that data, then it's yours. Do what you want with it.
So that's just generally how I see it is - either you're paying somebody for it, or at the very least you're crediting. And even if the crediting is imperfect, it's much better than having nothing.
So you feel like crediting and looking at similarity after the fact is maybe more feasible, or a faster solution, than trying to trace data provenance from the beginning?
Definitely. Because provenance from the beginning is - that's again, I think, where understanding how these work technically can be big deals. There are people who claim that these things are just plagiarism machines. They'll just spit out their training data. Mathematically not true. Very easy to disprove. And when this becomes a dominant narrative, then for a company like OpenAI, they just have to beat this narrative. And they can claim that, “Oh, look. You made this claim about our not working. Well, and we did do this thing.” And suddenly, our actual problem’s with getting rid of the following a false prophet. That, I think, happens a lot more frequently with tech than people realize.
So that's where understanding limitations becomes an issue. It's very hard for me to tell you which data point was more important or less important in most cases. Some very, very specific niche cases, sure, I might be able to point that exactly. But not necessarily so for the large, large majority. So in those cases, I really, really want to be careful. And just, like, there's something that's imperfect, feasible, and that still might benefit people a little bit, as opposed to trying to go for a perfect solution that will probably never happen and will just be much harder to pull off.
So you mentioned that you've built several AI-based tools and systems. I'm wondering, when you've built those systems, where has the data come from that you've used for building them? For instance, you mentioned the legal and the citation database. I am assuming that database of citations was acquired from a client? Or where does your data normally come from?
Most of the times, people give us the data to work with. Most of our solutions, AI is not just large language models. There's a bunch of other stuff that goes on. Stuff like, “Can I predict whether this would be a good customer for me?” “Can I predict how much time and effort I should put into following up on a particular lead?” But the different business profiles might have different kinds of lead follow-ups.
The Parkinson's disease algorithm that I mentioned earlier has nothing to do with generative AI. So most of the times, you have both publicly-available data, and also clients just give you data that you will work to build on. It's not as though I need to scrape the Internet to solve that really little problem.
There's also enough freely available datasets online. For instance, open Court Listener, et cetera, give you a lot of cases, et cetera, freely because they have done some great work in democratizing legal. So we can build on top of them fairly well. And that ends up forming a very strong base.
For the Parkinson's application, that's a really interesting one. I know some people with Parkinson's. Once you know they have it, it's easy, I think, to hear it in their voices. But for actually building a model that would predict it, you would have to have the voice recordings, and you would have to have some information about their medical diagnosis, perhaps. So that data seems more sensitive. How was that sourced for the development of the model?
So the baseline dataset we got was available publicly. Ironically enough, it was Apple who had done this research way, way back. So you don't have the voice recordings themselves. They had the features they extracted from the voices, and whether or not that voice had Parkinson's or not. So that tells us, “Okay, what are the features we should be looking for?”
And then what our team did was, we built on top of it. So we got additional recordings locally, and we did a bunch of voice perturbations too. Like, okay, delete a few values, have a few sorts of randomness here and there, and that creates a new dataset.
So, in that case, yes, the underlying dataset is very sensitive, i.e., whose voice is this? But I've taken away all personal information. If I just gave you a set of features of somebody's voice with no personally identifiable information, and said, “This person has Parkinson's. This person does not have Parkinson's.” And it's not demographic information. It's stuff like, “Oh, this person has, like, this is how much variance was in the sounds. This is the frequency average between that time period.”
One of the biggest predictors of Parkinson's that we found out is just, there's an instability to your voice, because your vocal cords can get, well, strained. So we have ways we can quantify that instability, by listening to the vocal pitch, that people can't really pick up on because we don't have that advanced hearing. But a machine, which can get very, very, good even on very, very simplistic forms, can actually record that audio and then just breaking that down into a spectrogram and whatnot, can make this available. So that's all stuff that we have done.
Ethically sourcing that data, again, if you really want to find it, generally, it's not hard to find. For most AI projects, good available data is not hard to find.
At the scale at which OpenAI and Anthropic and all operate, that becomes a different issue. Like, I could go and ask everybody who gave me their voice for the Parkinson's disease data set to do it. I don't think Sam Altman can do the same for OpenAI's data.
So that's where the unethical data becomes such a problem now. AI and machine learning have existed for many, many years. But for the most bit, the scale of where they were operating, they were operating in a way where it didn't have to. It's only the more recent technologies that things become more of a concern.
So whatever happened with the tool that you developed for detecting Parkinson's from voices?
So somebody reached out to us, and we commercialized that patent. They gave us a certain stipend, and they said, “Well, we get exclusive license to work and download and build on that algorithm.” So people have that now. And hopefully, they're actually building on it.
Oh, that's interesting. So they would end up making it available to doctors as a diagnostic tool? Or would they be marketing it to anybody - I want to find out if my voice has signs of early Parkinson's or not?
We were told that it would be made easily accessible to the masses. Now whether or not that happens, it hasn't happened so far, but hopefully that happens sometime now.
We didn't build the whole tool. We built the algorithm that had very good performance in those contexts. You know, low resources and high noise. Now to take that algorithm and actually put it onto a cell phone is a whole other issue.
This is one of those things that people don't understand with technology. It's very, very easy for me to run that algorithm on our Zoom recording on my laptop. But processing that on my phone becomes a whole different challenge - just the way it stores data, the computational resources available, whatnot. They become very, very different. So they've taken it. Hopefully, they build on it. If they do, I'll be sure to share them, and let everybody know that I saved the world! So I deserve a Nobel Prize! But we'll find out.
Yes. Yeah. It's very interesting to see the tools that have diagnoses like this. Even in a really, really good tool, there's going to be some percentage of false positives and false negatives, and those would have implications for someone who just runs a tool on their cell phone and thinks they have Parkinson's when they don't. Or what happens if their insurance company finds out about that diagnosis coming from that tool when it's not actually correct? So there's a lot of implications from it. It's one thing to get the technology to work. It's another thing to make it work well in society.
Yes. That's very true.
Well, hopefully, the company's dealing ethically with that, and working to make it something that adds value to the world.
Yeah. I hope so too.
Yeah. It's a very cool algorithm, though!
Because I'm very proud of it.
Yes! Alright. So I'm curious, are there any times where companies have used your data, or your content that you've created, for training an AI or machine learning system, and what you think about when that has happened?
Yeah. A few times, I've been told by the people that they found me because they Googled. They put, as I mentioned, using ChatGPT as Google, this ChatGPT “best AI newsletters”. And my newsletter happened to pop up, which means that it does know who I am, or it has that recognition. Or Gemini, for example. I don't remember what exactly this was, but I think it was my deep fake work, where I asked something, and it actually cited me to me. That was pretty cool to see.
And in those situations, yes, I am actually a lot more lenient with the whole compensation / credit thing with language models, as opposed to the generators. Because Gemini, ChatGPT, are available free for people. I do get some compensation for it, because they use my work, and now I get to use their solutions for free. Obviously, there's a subscription for the more advanced stuff, but the free tools are very, very good. So I don't actually mind them using my tool.
And the whole ethos around my work has been open source always. Open sourcing is, you can use my work if you want. That's why I don't have paywalls. That's why I have ‘If you want to share this, share this. If you want to translate and put this in your language, put this.’ I'm okay with it. So I'm not as worried about them. So far, I'm not that against them, specifically from the language model perspective.
And, also, if image generators were easily available and free to use, for everybody, then I think I'd be a lot more lenient with them on that perspective as well. Because I think, if they are free to use, then you get the compensation that way, that I can build on this. I can speed up. I can do better quality work because of it.
You mentioned earlier about companies that are capturing data and surveillance. Do you want to talk a little bit about that?
Yeah. That's definitely something I'm always going to be against. They can scrape social media feeds to build up, like, facial recognition platforms that can then be tracked. And often these things don't even work well. So they'll have a lot of false positives that have really bad justice consequences, because the wrong guy gets arrested, the wrong guy gets flagged, and whatnot.
But even if they work as intended, I think that itself is a big risk to democracies and freedom of people everywhere. Not everybody feels the same. A lot of people believe that, “With these surveillance technologies, you could have more safety, and you can identify criminals better.” Which I do agree with, but it's just, how much will it be used for that? How much will it be used to identify and potentially go after protesters? Or how much will the threat of that stop active and public speaking out against injustices? Because suddenly you know that there's always these tools that are watching you and that can identify you.
So that's always my concern, and this is where I think I would like to point out that I don't have the best insight for this. I have a certain belief on this. I might be completely wrong. Maybe these will be much, much more beneficial to society than they will be harmful. But I just have that natural skepticism around it.
Well, I think that skepticism is pretty well justified. We've seen a lot of cases with companies collecting data without permission and using it for purposes that were never intended or never consented to. So I think it's very sensible to be skeptical about it.
I’ll say yeah.
So, looking at everything that we've talked about, there's certainly a lot of distrust of the AI and tech companies and the way that they use our data. And I'm wondering what you think is the most important thing that they could do, or that we could make them do, that would reestablish that trust going forward? If you have any specific ideas on how we can do that?
As companies or as individuals? Like, what we can do?
Either.
I think citations will be a big credit aspect of it. Because, again, even if it's incomplete, I think that's still huge, and largely beneficial to people.
As people, the most important bit is always to prioritize. You know, you can't fight every battle. So you have to pick what you believe is important, or the most important. Parts that you can care about. And then you kind of have to become heartless towards everything else. Focus on this and develop your skills enough to where you can make a difference.
Because, as you said, we have a lot more power than people think. But I think people aren't using that yet. Like I mentioned, adversarial perturbation. I’ve been very against, like, governments tracking the Internet to stop access to information. So that's one part.
Another thing that I cover, but I don't think that many other people care about, which is: how do you break automated weapon? And, like, how do you break automated government censors? How do you break signal jamming and whatnot? That's all stuff that we have covered, that I continue to cover regularly.
So you do have to pick what you believe is important and what you can make a difference in. And then focus on, “Okay, this is where the injustices that happen, and what can we do”, to “How can I become competent enough to actually meaningfully hit them where it hurts, as opposed to doing it more performatively?”
Obviously, there's a lot of downsides to AI. And there's obviously also a lot of upsides. If you think about the positive side for a minute, the positive benefits for society, what would you say is the one most positive benefit of AI that you would call out, that it's the one thing that you want to ensure we don't lose as we try to address the harms?
The same thing that makes AI dangerous is also what makes it very useful, which is that it can scale up things, operations at a way that's never done before. Mentioning IQIDIS earlier, you know, we're now speeding up people's legal workflows by, like, 50, 60%, which means that what would have cost people days to do now takes them hours, which is massive for us. Imagine a legal fee that doesn't cost as much, because your lawyers are able to do much more, efficiently.
Parkinson's disease, same story. Because we scaled it up in no-resource environments, suddenly, people that don't have good cell phones are also able to get that possible diagnosis early.
Other diagnostic AI done well has a similar benefit, which is: even if it doesn't work well 20% of the times - which is a huge, huge error rate in anything technical - if I can save 4 out of 5 people, that's it. I'll take that trade, as long as there are no downsides to the other 1 [of 5].
I don't like the HR stuff because there you're really, really hurting the one person. I want to be very, very careful when I'm using AI for credit score analysis. It can be done to make it more efficient, but you want to be very, very careful on what kinds of data you're looking at. Because you can also really, really hurt that one person who comes from a not-great background, who might come from demographically backwards groups. And then suddenly that AI looks at those data points and makes a decision that harms them.
So AI will reinforce biases - biases from the information theory perspective of shortcuts to evaluating decisions. You want to be absolutely confident. Where do you not mind a bias being reinforced? I don't mind a bias being reinforced to do some work quicker that's being done now. I don't want a bias being reinforced where that might really hurt somebody unfairly, i.e., somebody comes from a backward ethnic group, and that bias gets reinforced to reject them on the basis of that.
That really is how I evaluate and see AI. So it can do things very effectively. I just hope it's done in a way that's useful and not harmful.
Yeah. I agree. I think, looking at the consequences of the false positives and the false negatives, and especially in cases where people normally don't have any recourse: they might not even know that an AI algorithm scored them, and then it came up with this recommendation based on biased data. They may not have any way of knowing that. So any use of it for those kinds of purposes where the person doesn't know about it, or can't do anything about it, or they have no way to appeal it, those are definitely problematic. But like you said, if it’s 80% help rate, where there's no help right now, it’s a pretty good win.
Yeah. Exactly. That's really it, is, you definitely don't want to “throw out the baby with the bathwater”. We want to be aware that there are always downsides, and there will be mistakes where humans, we will screw up, tremendously even. But we want to ensure that our systems are set up in a way that we never make the same mistake twice.
Well, Devansh, thank you so much for making the time to share your insights about AI with me today. Is there anything else that you'd like to share with our audience?
I think just generally, to reiterate on a point that we talked about - there is a lot more that you can do than people realize. And a lot of the AI hype cycle, it predicates on the fact of putting fear in you. Like, make you feel like you're falling behind, scared of the technology, or you have to buy if you want to keep up. Both of those are problematic.
And as long as you work on the fundamentals and keep developing your skills, I think you'll be in a much better position than you would originally think. Things aren't as doom and gloom, or completely about to change and revolutionize us, as some people pretend that they are.
Because of the hype, people will often put a lot more credence into what people are saying than they should have. Don't take our words fully, because there's so much we don't understand. But that also comes to the fact that there's just a lot of incentives that can often go wrong. There are a lot of other areas. So the world's a lot more open, and there's a lot more to be done than sometimes is pretended. And there's a lot more that people can do than sometimes is fully appreciated.
Well, thank you so much for joining me for this interview today, Devansh. It was a lot of fun, and I enjoyed getting to meet you and get to know about some of your experiences. Really cool work that you've done there. I'm looking forward to continuing to read your writing about it.
Thank you.
Interview References and Links
Devansh on LinkedIn
Devansh on Substack (@chocolatemilkcultleader)
About this interview series and newsletter
This post is part of our AI6P interview series on “AI, Software, and Wetware”. It showcases how real people around the world are using their wetware (brains and human intelligence) with AI-based software tools, or are being affected by AI.
And we’re all being affected by AI nowadays in our daily lives, perhaps more than we realize. For some examples, see post “But I Don’t Use AI”:
We want to hear from a diverse pool of people worldwide in a variety of roles. (No technical experience with AI is required.) If you’re interested in being a featured interview guest, anonymous or with credit, please get in touch! (And check out our guest FAQ)
6 'P's in AI Pods (AI6P) is a 100% reader-supported publication. (No ads, no affiliate links, no paywalls on new posts). All new posts are FREE to read and listen to. To automatically receive new AI6P posts and support our work, consider becoming a subscriber (it’s free)!
Series Credits and References
Audio Sound Effect from Pixabay
If you enjoyed this interview, my guest and I would love to have your support via a heart, share, restack, or Note! One-time tips or voluntary donations via paid subscription are always welcome and appreciated, too 😊
