2022-03-17 18:07:50 RT @trackingexposed: Today we release a special 24 page report into TikTok’s activities in Russia. Our researchers exclusively discovered t… 2022-03-16 02:08:01 Not to mention all the ethically questionable endeavors... 2022-03-16 02:07:21 Task design is such a crucial part of evaluation. I genuinely think a lot of ML failures are just the result of ML methods being thrown at tasks that are completely inappropriate for ML to solve (ie. data with few examples, sparse features, unknown & 2022-03-15 22:58:04 @PreetumNakkiran @realCamelCase 2022-03-15 14:32:58 Every time something goes wrong with an ML model in deployment, researchers in our field are quick to call it "distribution shift", but that term is both too broad & Excited to blog w/ @beenwrekt about what ML has to learn from other fields on this! https://t.co/1RONiqMNJk 2022-03-14 17:07:36 RT @KateKayeReports: For the third time, @FTC has forced a company to destroy algorithms built with data gathered deceptively. Here's why i… 2022-03-12 18:41:23 RT @sherrying: @crystaljjlee I was just thinking about this. funny how companies chose to say trustworthy AI and responsible AI and none ch… 2022-03-12 18:38:20 @_alialkhatib @crystaljjlee Completely agree with this! I now overwhelmingly use "harm" as well. 2022-03-12 08:11:00 CAFIAC FIX 2022-01-25 10:29:05 RT @alexhanna: @ChristophMolnar We have a whole research program dedicated to understanding this... https://t.co/uJXeJepliQ https://t.co/P… 2022-01-25 10:16:21 @alexhanna @ChristophMolnar Though I'm glad to see the fiasco of faulty covid-19 work has begun to wake a lot of people up to the reality of how bad things are. We need even more people speaking up about this, and calling it out in order for things to actually change for the better. 2022-01-25 10:10:00 @alexhanna @ChristophMolnar Yep, and lots of even earlier conversation here: https://t.co/DriTstdhTl 2022-01-24 22:20:39 RT @daniel_d_kang: ML models are being deployed in mission-critical settings, such as autonomous vehicles. Shockingly, the data used to tra… 2022-01-24 15:35:26 @MarkSendak Yeah, totally agree. I'm of the opinion that ML is actually really useful, but not in the end-to-end way it's being advertised as useful. It's often only likely to solve one specific type of sub-problem. 2022-01-24 15:20:21 @MarkSendak They sold the vision of an AI doctor based on an unrelated game demo - in reality, the problems they would need to solve were more complex & 2022-01-24 15:16:24 @MarkSendak Hm that kind of info recall is great for passing med school but is it all that's necessary to be a good doctor? Workflow integration was a big part of the problem precisely because they failed to honestly break down what ML could or couldn't do, & 2022-01-24 15:11:17 Whenever I bring up issues, I hear "Well ofc nothing works irl." ...What? What is the point of these evaluations in research if not to indicate something meaningful about real world performance? This paradigm gap threatens the integrity & 2022-01-24 14:57:27 @LeonDerczynski Yep + I think it boils down to a combo of 1) underestimating the complexity of the real problem & @ChristophMolnar has been tweeting a lot about the COVID mess recently & 2022-01-24 14:45:35 yes, this is a subtweet on recent news: https://t.co/Y0p8DpvoW4 2022-01-24 14:45:02 Case in point: what about solving Jeopardy! made us think IBM Watson would succeed in healthcare? https://t.co/7YFTeblAbA 2022-01-23 16:20:14 RT @yaroslavvb: Table 2 of https://t.co/yy2MI0M6Tz shows what's wrong with ML research. Papers got in by providing a theorem (checked by re… 2022-01-18 15:49:58 RT @etechbrew: .@mozilla fellow @rajiinio. https://t.co/AHyrp2OAbh https://t.co/OPs2Aoq9FT 2022-01-17 08:11:00 CAFIAC FIX 2022-01-12 18:26:53 RT @hypervisible: “…we must be honest about what can realistically be accomplished by these piecemeal attempts at stitching sociocultural e… 2022-01-12 16:12:14 @Abebab lol abeba it is still due eventually 2022-01-12 16:12:02 RT @FAccTConference: Due to recent disruptions & 2022-01-12 16:07:42 @KLdivergence not the hero we deserved but the hero we needed 2022-01-12 15:10:01 I like the paper so much because this is a common experience with auditing - you may request info & 2022-01-12 15:10:00 One of my fave papers on algorithmic transparency is "Seeing without knowing" by @ananny & 2022-01-12 14:58:46 RT @NaDomagala: Calling the AI ethics crowd - I'm compiling a reading list on #algorithmictransparency and I need more global examples & 2022-01-12 04:30:47 @jovialjoy @MIT @GeorgiaTech @UniofOxford @EmoryUniversity @medialab @EthanZ @LatanyaSweeney @kanarinka @AJLUnited CONGRATS JOY It's Dr. Poet of Code now! 2022-01-12 00:48:07 RT @tdietterich: The term "ablation" is widely misused lately in ML papers. An ablation is a removal: you REMOVE some component of the syst… 2022-01-11 08:11:00 CAFIAC FIX 2022-01-06 05:06:39 @Miles_Brundage @STS_News Also, the "The attachments of ‘autonomous’ vehicles" is a really level-headed description of what's happening with self-driving cars right now. Convinced me that no one will be driving self-driving cars if we don't fix safety issues. https://t.co/x67ySaILrK 2022-01-06 05:02:10 @Miles_Brundage Yeah - "Moving Violations" from @STS_News makes a version of this argument well (ie. regulating safety issues allowed the industry to exist/innovate) https://t.co/TBS18w6EY6 2022-01-06 04:48:45 @BlancheMinerva @jackclarkSF @mer__edith The large scale projects he references in those areas are for data collection - the large scale projects he's proposing for AI/ML aren't data collection instruments but models. That requires a different type of tooling, which I'm personally not convinced needs to be large scale. 2022-01-06 04:45:25 @BlancheMinerva @jackclarkSF @mer__edith Meanwhile academic teams are much harder to maintain. An individual could graduate & 2022-01-06 04:42:37 @BlancheMinerva @jackclarkSF @mer__edith I'm not talking about an individual person but the existence of a team... The tensorflow team has been intact for a while at this point and that means dedicated resources in any given year to propping up the infra necessary for the framework to remain effective. 2022-01-06 04:40:08 @Miles_Brundage The economic argument for safety (vs just discussing individual/community harm) is a huge theme in earlyish automobile industry. Interestingly, that language is kind of popping up again with concerns of safety issues and accidents slowing down the adoption of self driving cars. 2022-01-06 04:27:32 @jackclarkSF @mer__edith It's cool to think about what a common project would look like, that wasn't tied to a single grad student but developed across various labs. Here's where lessons from LHC and other large collaborative efforts could become a really useful model. 2022-01-06 04:26:03 @jackclarkSF @mer__edith Yeah this is a really interesting point. Academic open source projects are hard to manage and maintain - people graduate etc. - while companies pretty much have a steady team and stream of resources propping the tools up for years. 2022-01-06 04:20:21 @jackclarkSF @mer__edith By the way, I think you're totally asking the right questions here. The analogy to other large scale scientific instruments is an interesting one and worth thinking about. Just pointing out some of the reasons why I think the AI/ML space may be different. 2022-01-06 04:18:21 @jackclarkSF @mer__edith Hm, but this "if they build it, they will come" mindset assumes it's a good idea to build it. It's fine to be convinced about the importance of large AI models but not everyone is. Choosing to invest in that direction as an org is very diff from directing public funds there. 2022-01-06 04:11:44 @jackclarkSF @mer__edith Yeah I totally agree. But unless you have access to the training set, there's no way to be sure that the public model is going to behave even remotely similar to any other models the researcher is actually trying to understand. 2022-01-06 04:04:06 @jackclarkSF @mer__edith Though for the latter, I don't know if the tradeoff of dumping your data on some shared machine is worth it for most people (privacy, etc). It would be much more helpful to get funding directly to set up or make decisions about whatever infrastructure the researcher sees as best. 2022-01-06 04:01:57 @jackclarkSF @mer__edith Though we could make Google build their model on public infrastructure and that could be interesting. And yeah, I personally think the most potentially interesting public models are not going to be these large scale things. Ex., government & 2022-01-06 03:57:14 @jackclarkSF @mer__edith I don't think either of these assumptions hold. The large AI model built on public infrastructure is not going to be the same one that Google builds and there's a reason it could be more meaningful to poke at Google's model (for example, bc it affects users and non-users). 2022-01-06 03:55:15 @jackclarkSF @mer__edith hm, there's a couple assumptions happening here - that all large AI models are made equal (which we already know not to be the case) and are thus equally meaningful or interesting to probe, and that there's enough researchers interested in thinking about these kinds of models. 2022-01-06 03:47:09 RT @math_rachel: - Annotator disagreements may capture important nuances ignored by a single ground truth - A multi-task based approach yie… 2022-01-06 03:41:41 @jackclarkSF @mer__edith Some things I see no one talking about that I'd want to see more discussion about though: Frameworks - all our models are trained on pytorch, tensorflow, etc. Is there value in a non corporate framework for model training? What's the value of common infra vs increased funding? 2022-01-06 03:37:45 @jackclarkSF @mer__edith Making those judgements as a company feels fine (ie your product, your choices), but as a "collective" resource it'll be difficult to build models/datasets that most people will see as meaningful or unproblematic. Even the direction of "let's build larger models" is contestable. 2022-01-06 03:28:34 @jackclarkSF @mer__edith For instance, to collect the data to answer specific questions in physics we needed the LHC, but to build some common GTP-x requires a lot of contextual value judgements & 2022-01-06 03:21:54 @jackclarkSF @mer__edith Even some things that it seems we all think we need (ie compute, data) requires a lot of value judgements and the person controlling the resources will be making a lot of those decisions (perhaps inappropriately) on behalf of others. 2022-01-06 03:19:47 @jackclarkSF @mer__edith AI/ML research problems require eng infra around data analysis, processing, evaluation and not just collecting data. So the tools need to be more context specific and it's unclear if there are actually common resources that would be helpful here. 2022-01-06 03:17:21 @jackclarkSF @mer__edith This is a really interesting question to ask! My personal take is that there are characteristics of CS research and AI/ML work in particular that makes such large scale engineering work not feel worth it. These other projects are about eng infra for data collection processes but 2022-01-05 23:12:44 @KLdivergence Oh Noo 2022-01-04 18:53:11 @mariafarrell @IfeomaOzoma @erikashimizu Thanks for sharing this! By the way, I'm a big fan of your work, I really enjoyed this past article: https://t.co/QlJhHbtwjq 2022-01-04 17:08:14 @NThylstrup @Diogo_PH22 @IfeomaOzoma @erikashimizu @Agos_Daniella This looks great - thanks for sharing! 2022-01-04 17:00:08 It's been heartbreaking to see talented female researchers being gaslit about their own competence. It's really Pedro and his like that should be the ones second guessing and asking themselves "Did I get this opportunity only because I am a white man?" https://t.co/oV5aLGfvRf 2022-01-04 16:48:59 Like, I really wish anyone gave me anything just for being a Black woman. Then maybe there'd be more than one or two of us. 2022-01-04 16:48:58 It's so ridiculous to me when a white guy says that a minority only got an opportunity because they are a minority,... that guy is literally standing on centuries of white guys like him only being given opportunities because they are white men (as everyone else was excluded). 2022-01-04 16:23:38 RT @leonieclaude: #Digitalisierung wird von Männern geprägt – ist oft zu hören. Doch das ist nur die halbe Wahrheit. Denn die Silicon-Valle… 2022-01-04 16:22:07 @Diogo_PH22 @IfeomaOzoma @erikashimizu Yeah, true. Visibility is definitely a double-edged sword. But I don't think this is a reason to exclude minoritized perspectives - in fact, it's all the more reason to protect whistleblowers as we elevate them. 2022-01-04 16:05:23 It really does matter who we elevate. For eg., I'm glad @IfeomaOzoma & 2022-01-04 15:45:53 I've already noticed something similar with the Facebook case, where Frances Haugen is much more visible in many ways than Sophie Zhang, even though the latter is more intimately familiar with the specific policy & https://t.co/PNVMsfLv56 2022-01-04 15:30:55 Whistleblower politics is so weird. I can't help notice who the press & 2022-01-04 15:06:49 RT @FAccTConference: Two major #FAccT deadlines coming up: (1) Final submission deadline for PAPERS: Jan 14 (https://t.co/t2g275CtOK)… 2022-01-04 15:06:34 @alexhanna Oh no :( 2022-01-04 15:00:18 RT @leonieclaude: Die nigerianisch-kanadische Forscherin @rajiinio überprüft Software, die das Leben von Menschen zerstören kann. Das kommt… 2022-01-04 15:00:14 Thanks so much Roberta for featuring my work here! Such thorough reporting :) https://t.co/EbWJcHLUzM 2022-01-04 14:58:10 RT @AnnaNosthoff: Beeindruckender Auftakt zum Jahresbeginn heute @republikmagazin: @leonieclaude porträtiert in der Serie "Digital Warri… 2022-01-03 15:09:47 @holdmytowel Perhaps those are the incentives to produce this kind of work but it's pretty strange to me that the reviewers (for both funders and conferences) don't currently have the assessment tools to call the bluff. 2022-01-03 04:48:48 @tdietterich hm, I think it's a combo of this & 2022-01-03 04:35:23 It's frustrating because you can't necessarily blame individual authors for this - they're just following flawed evaluation norms, and are rewarded for this with publication. It's a community-wide issue. We kind of start the conversation in this paper: https://t.co/l8jRwE4HL7 2022-01-03 04:29:55 It's a serious problem & All illusionary progress. 2022-01-03 04:26:35 It's pretty remarkable how many published ML papers end up being practically & Evaluation practice defines what is accepted as evidence of progress in a field - ML's incoherent evals have clearly skewed our judgement of what a valid contribution is. https://t.co/K3UZJQ4MiQ 2021-12-31 22:55:19 RT @aylin_cim: Researchers and practitioners interested in fairness, accountability and transparency in socio-technical systems: We look fo… 2021-12-27 08:20:00 CAFIAC FIX 2021-12-21 08:21:11 @AmandaAskell But we don't design children, at least not directly in the way we do engineered objects like AI. Even when things are complex, it'll still be on those that made the decisions leading to the agent's outcomes that should be held responsible. Blaming "AI" really means blaming "ppl". 2021-12-21 08:08:57 @AmandaAskell But this is not a birth, it's a build? I'm not sue why we would humanize anything other than the humans... Engineered artifacts (from a bridge to an AI system) are a direct consequence of design choices, not the mysterious result of a natural process where design is not a factor. 2021-12-21 07:58:34 @AmandaAskell I see an AI system the same way I see I toaster or a car, and don't see any reason or benefit to projecting human characteristics unto it. 2021-12-21 07:55:19 @AmandaAskell Perhaps we can agree to disagree here. I consider that analogy to be appropriate bc there are practical limits to what one can blame a parent for due to a child's agency, but as a constructed artifact, responsibility is framed directly as consequences of eng choices wrt outcomes. 2021-12-21 07:30:40 @AmandaAskell Here I point to the irony of the fact that in order to argue for the humanization of AI, people will actually de-humanize the real people in the process - this includes people embedded in the data, but I'm beginning to realize this extends to others (annotators, impacted, etc.) 2021-12-21 07:27:47 @AmandaAskell Humanizing AI inappropriately has repercussions - specifically, I believe it leads to a false sense of absolved responsibility in those building AI systems (ie. treating AI as an independent agent, like a misbehaving child, and not the built artifact it is, disguises eng choices) 2021-12-21 05:47:51 This old tweet feeling more and more relevant these days... https://t.co/Pe0KsykYvp 2021-12-20 19:33:14 Excited to connect with colleagues at University of Toronto today, to discuss algorithmic auditing & It's been great to see how the work on that has evolved since I was a student there! https://t.co/9jL4BHUlJA 2021-12-18 00:33:09 @rasbt @thegautamkamath Interesting. This reminds me of @swabhz's paper: https://t.co/ez8IWWpuC9 2021-12-18 00:28:12 @mmitchell_ai Email me! (+ So excited you are teaching this class!) 2021-12-17 22:53:43 RT @WHOSTP: JUST LAUNCHED: The AI Researchers Portal on https://t.co/2aGbEHpYgP. It’s a central connection to many Federally-supported reso… 2021-12-17 20:28:51 RT @GoogleAI: Dataset distillation enables #ML models to be trained using less data and compute. Today we introduce two novel dataset disti… 2021-12-17 19:14:26 @o_saja @SmithaMilli Totally agreed - I also find that a lot of the essential "data in ML" references come from CHI, CSCW and CSS venues! 2021-12-17 18:59:48 RT @MaxALittle: Evidence for the claim: "deep learning (AI) has solved vision" is based heavily in ImageNet/CIFAR classification accuracy.… 2021-12-17 18:56:55 @o_saja Yes, I think this "big data" era work is still totally relevant. Class 1 could be a reading of "10 Rules for Big Data" (https://t.co/BUjR2aOM3L) + Dotan & 2021-12-17 18:52:42 @Aaron_Horowitz Hey, I said ONE day, not TOday 2021-12-17 18:22:32 I really shocked myself this morning, realizing I can cite a whole syllabus worth of "data use in ML" references off the top of my head...lol someone needs to let me teach this class one day, it would honestly be so much fun! https://t.co/rQPPEIDF2Q 2021-12-17 18:18:51 @thegautamkamath @emilymbender @ergodicwalk @amandalynneP @cephaloponderer @alexhanna @timnitGebru @laroyo @o_saja @harini824 @morganklauss @amironesei ok actually, one last thing bc I don't think this paper got enough love & -> + related papers by @vinodkpg et al.: https://t.co/f9zGQs8zTk,  https://t.co/5HsKLfn2aC 2021-12-17 18:09:31 RT @Abebab: This book is finally out and I am super excited about it. In the midst of so much AI over-hype, over-promise, and unsubstanti… 2021-12-17 18:07:49 @IasonGabriel 2021-12-17 18:06:06 @thegautamkamath @emilymbender @ergodicwalk @amandalynneP @cephaloponderer @alexhanna @timnitGebru @laroyo @o_saja And @harini824's work on mapping biases: https://t.co/LTm0qlNSER And @cephaloponderer, @alexhanna, @morganklauss & Aha let me stop here, I could honestly go on and on about this! 2021-12-17 18:02:53 @thegautamkamath @emilymbender @ergodicwalk @amandalynneP @cephaloponderer @alexhanna @timnitGebru @laroyo And I like @o_saja's work on taxonomizing data bias and approaching that methodologically: https://t.co/Ohyc9NgJHl 2021-12-17 18:00:24 @thegautamkamath @emilymbender @ergodicwalk @amandalynneP @cephaloponderer @alexhanna @timnitGebru @laroyo + https://t.co/pEo1rkicQW ( and earlier work on crowdsourcing - https://t.co/0V3Shnb19d) 2021-12-17 17:59:27 @thegautamkamath @emilymbender @ergodicwalk @amandalynneP @cephaloponderer @alexhanna @timnitGebru I especially learnt a lot from "Lessons from the archives". Also a lot of @laroyo's work is quite practical: https://t.co/9d6svMoCQs 2021-12-17 17:55:26 @thegautamkamath @emilymbender @ergodicwalk @amandalynneP @cephaloponderer @alexhanna Methodology-wise, a lot of @timnitGebru's work will be relevant (& https://t.co/Dq8zzGgtM3, https://t.co/QQ1I2uQHcj, etc. 2021-12-16 13:40:25 RT @TechCrunch: France latest to slap Clearview AI with order to delete data https://t.co/6Tg9ZXEn3R by @riptari 2021-12-16 03:31:22 RT @chopracfpb: Today, the @CFPB is re-launching its whistleblower website based on user research with people in the tech industry. Submiss… 2021-12-16 02:18:15 RT @Damini_Satija: Job opps! Today we’re launching the first two openings @AmnestyTech’s new Algorithmic Accountability Lab (AAL) 1)… 2021-12-15 20:22:23 RT @cfiesler: So apparently one of @FortuneMagazine's highlights of the entire #NeurIPS2021 AI conference was me throwing shade on Kant. ht… 2021-12-15 20:20:56 RT @AIESConf: Apologies for the silence as we tried to figure out how to maximize the chance of a physical conference... We're planning to… 2021-12-15 16:24:03 RT @FAccTConference: REMINDER: THIS IS DUE TOMORROW! Please get your abstract submissions in 2021-12-15 14:23:09 RT @Damini_Satija: Some news! So happy to have joined the @AmnestyTech team to head up a new Algorithmic Accountability Lab. Feels like a m… 2021-12-15 13:49:35 @Abebab @amandalynneP @red_abebe can't wait to read you attempting to be optimistic about AI 2021-12-15 13:48:39 @Abebab 2021-12-15 02:40:09 @Abebab @amandalynneP @red_abebe Grateful for the scholarship of those dedicated to the work - it inspires & I don't think we celebrate enough how amazing it is to have such role models available - I'm so happy to learn from all of you! 2021-12-15 02:36:39 Recently had the joy of watching the likes of @Abebab & People like to complain about the chaotic state of the field, but - I really just have so much hope for the next generation of scholars in this space. 2021-12-15 01:40:51 For various reasons, I have withdrawn from this event, and will no longer be participating in the debate. I know how hard it can be to organize events like this, so I do not make such a decision lightly, and I truly do thank the organizers for inviting me! https://t.co/3bN1KHVKcl 2021-12-15 00:51:47 RT @DavidVidalAI: Proud to be a part of this important collaborative effort with @MarkSendak, @rajiinio, and other amazing individuals http… 2021-12-14 21:21:10 @sociotech_jrd @schock @timnitGebru @bsmith13 @jovialjoy This is a main theme of @PopTechWorks's Automating Inequality! 2021-12-14 16:54:06 https://t.co/jwNCvVLrfY 2021-12-14 12:58:06 RT @jovialjoy: Thank you @TheKingCenter for this amazing honor and for elevating the work of @AJLUnited. I cannot wait for the beloved comm… 2021-12-14 12:54:25 RT @meatspacepress: Fake AI is out now! The book contains seventeen chapters charting the contemporary twists and turns of AI hype, pseudo… 2021-12-14 12:54:15 RT @FAccTConference: Do you have research that needs to have wider reach because of its importance to the daily lives of people & 2021-12-14 12:54:12 RT @FAccTConference: Our Call for Applications for Diversity, Equity and Inclusion scholars is now open! Funding up to $25,000, project len… 2021-12-11 15:36:52 RT @annargrs: A great discussion of ethics checklists! A highlight from the panel: Q (@emilymbender): ethics checklists have the danger of… 2021-12-11 15:34:27 @perayson @JesseDodge A recording is available here: https://t.co/yFdpVv5YH6 Right now I think you still need to register to see it, but hopefully the video will become public soon! 2021-12-11 15:33:04 @agstrait Ah, seems a recording is still accessible here if you can register (I'll flag if it becomes public later!): https://t.co/yFdpVv5YH6 2021-12-11 15:30:12 @agstrait Not sure :( Hoping Neurips will release its recorded content after the conference is over! 2021-12-11 00:16:43 The craziest part of this panel was when I was casually mentioning "The Checklist Manifesto" and @JesseDodge just pulls out the copy lying on his desk What an amazing group! Learnt so much from these panelists! https://t.co/l37lvjdAEj 2021-12-11 00:12:08 @Abebab Thanks for such amazing contributions 2021-12-11 00:09:28 @ruthstarkman @Abebab @cfiesler @JesseDodge Thanks for watching! 2021-12-10 22:06:54 @mmitchell_ai here! https://t.co/yFdpVv5YH6 2021-12-10 21:57:16 Excited to host this panel soon! There's so much to talk about when it comes to research ethics in the ML community, and I'm glad we could assemble this group of experts to approach the discussion from multiple angles, speaking to a broad range of ethical considerations. https://t.co/l37lviWxCj 2021-12-10 21:54:19 RT @JesseDodge: This is today! Looking forward to being on this plenary panel at NeurIPS to discuss ways we can build incentive structures… 2021-12-10 21:35:28 @Abebab 2021-12-10 21:27:36 @Abebab Lol thanks for doing it! 2021-12-10 21:24:31 RT @mirianfrsilva: To everyone who participated in #NeurIPS2021 and attended to @black_in_ai Workshop, thank you very much! We as organi… 2021-12-10 21:21:56 RT @stochastician: It's hard to overstate how much I love this paper: "Plenoxels: Radiance Fields without Neural Networks" https://t.co/vO… 2021-12-10 21:13:52 RT @vnasilva: Today is the day @black_in_ai program is live at @NeurIPSConf, come join us! https://t.co/bNHajKtcfs 2021-12-10 21:11:39 RT @alexhanna: @SashaMTL @rajiinio ICYMI -- @consentfultech, @Data4BlackLives, @DetCommTech, and @and_also_too just released this Consentfu… 2021-12-10 17:06:00 lol I do not look forward to having to write an "AI and Everything in the Whole Wide Metauniverse(s) Benchmark" paper in a couple decades https://t.co/f5bEIrtYvw 2021-12-10 16:43:50 RT @MarkSendak: (1/4) With support from @PJMFnd, we’re thrilled to launch a new collaboration between @DukeInnovate @DukeHeartCenter @Zaina… 2021-12-10 11:28:05 RT @timnitGebru: Watching this talk now, and no slides was 100% the way to go. I realize more than half of my slides are really not necessa… 2021-12-10 00:39:42 She killed this speech. What a great talk! https://t.co/19cmUV25oC 2021-12-10 00:39:14 RT @MadamePratolung: "The next frontier in databases is inside computers." Gender, Allyship & 2021-12-10 00:19:21 RT @MadamePratolung: @mathbabedotorg /11a @rajiinio we can't keep regulating AI as if it works Sect… 2021-12-09 17:05:00 RT @emilymbender: Poster session happening now! Come say hi :) #NeurIPS2021 2021-12-08 16:16:32 RT @dillonniederhut: Over time, papers tend to user fewer and fewer benchmarks for evaluation. This is a problem, because the results might… 2021-12-08 16:08:43 @laroyo @mrtz @sleepinyourhat @joavanschoren ends with a comment on how its really to the benefit of the community to address the benchmarking issues thoughtfully and ethically, and that it's to everyone's advantage to think about how to create and disseminate a diverse set of useful benchmark datasets. 2021-12-08 16:03:03 @laroyo @mrtz @sleepinyourhat @mrtz mentions the ethical issues associated with benchmarks. Isabelle says the need for accountability and reviewers scrutinizing the benchmarks before they enter mainstream use. @sleepinyourhat mentions crowdsourcing "labor nightmare" & 2021-12-08 15:58:23 @laroyo @mrtz @sleepinyourhat Lora: "One of the challenges is that we don't have any way to measure unknown unknowns - we can only measure what the model can see and understand. This can be quite dangerous in high stakes domain, so capturing these blind spots is very essential I think." 2021-12-08 15:57:01 @laroyo @mrtz @sleepinyourhat Mentions RL and how that became popular following the popularity of benchmark performance in game environment vs. live robot competitions at Neurips (where they were trained in simulation, and all failed miserably, actually) 2021-12-08 15:55:35 @laroyo @mrtz @sleepinyourhat "Not every interesting problem lends itself to a benchmark - in fact many problems don't. Often, interventions are needed to validate an approach, and are either unethical or too costly (ie. scientific discovery, theory, etc.) to make a data benchmark from." 2021-12-08 15:54:34 @laroyo @mrtz @sleepinyourhat "This ends up being more engineering than research and we want to avoid this as well." 2021-12-08 15:53:40 @laroyo @mrtz @sleepinyourhat Isabelle:"Hopefully we can have more principled approaches in how they are created and we have a proliferation of a greater number of benchmarks. Everyone is trying to imitate how to solve the problem in the same manner since they're all focused on the same benchmarks & 2021-12-08 15:52:16 @laroyo @mrtz @sleepinyourhat Isabelle: "People cherry-pick the benchmarks they want to work on & 2021-12-08 15:52:08 @laroyo @mrtz @sleepinyourhat @mrtz asks about "dynamic, open ended benchmarks" and what it would take to address these tasks. + "What are some research directions we miss out on because they do not fit this benchmarking paradigm?" 2021-12-08 15:49:52 @laroyo @mrtz asks about distribution shift and @sleepinyourhat says small domain-specific benchmarks that shift from the training data are a decent start but warns against the appeal of "adversarial benchmarks", since you can get to pretty consistent distortions in performance metrics 2021-12-08 15:48:18 This is the paper @laroyo mentions: "Truth Is a Lie: Crowd Truth and the Seven Myths of Human Annotation" https://t.co/0V3Shnb19d 2021-12-08 15:43:16 @mrtz asks about human annotators & 2021-12-08 15:40:43 Mentions "data excellence" framework to come up with a more robust set of practices around how we design datasets - inclu "maintainability of data at scale" (similar to software at scale), "validity" (capturing correlation between data and external measures, inspo from education) 2021-12-08 15:40:42 "We are testing accuracy on tasks with majority votes where there is genuine subjectivity in the results, and room for interpretation" @mrtz mentions human comparison, & 2021-12-08 15:32:54 @sleepinyourhat being very honest rn about NLP benchmark limitations - "We've got this discourse on what language models are good at that's not very grounded - you can point to benchmark performance to make claims, but also point to embarrassing failures & 2021-12-08 15:26:37 This is happening now! Amazing panel so far https://t.co/ZY8GrELczh 2021-12-08 15:23:12 Come talk with us about this during the poster session Thurs: https://t.co/CyteuTxr6x 2021-12-08 15:21:36 Great Question! @ThomasILiao will be presenting this paper at the Dataset and Benchmark Poster Session 3 Thu 9 Dec 8:30 a.m. PST — 10 a.m. PST. Please come check it out! https://t.co/llPAd9HOvJ https://t.co/ND8hIUJdkb 2021-12-08 06:11:43 @timnitGebru @rtaori13 @lschmidt3 @TheNapMinistry Thank you! I'm working on it! 2021-12-08 06:05:49 @timnitGebru @rtaori13 @lschmidt3 @TheNapMinistry Lol I nap regularly, don't worry! Everything gets released at the same time, but it's all a long time coming. 2021-12-08 03:32:54 There are *so* many things to think about when it comes to ML evaluation - some of which the field has yet to investigate properly! Glad I could be a part of this project, led by Thomas Liao, and with @rtaori13 & Paper here: https://t.co/uotjMGAcl2 https://t.co/vOF20UBDwe 2021-12-08 03:32:51 We reviewed 100+ ML survey papers & Often framed as a one-off casual consideration, ML eval is rarely presented as what it is - a chained *process*, rife w/ measurement hazards https://t.co/LT5f4TSDhN 2021-12-07 22:00:13 @cfiesler We are just as excited to hear you speak! 2021-12-07 21:59:43 RT @cfiesler: Typically plenary panels are a captive audience. I'm not sure how true that is for an online conference, but I am both excite… 2021-12-07 19:17:03 RT @CANSSIOntario: Deborah Raji, Fellow @Mozilla, speaks next @UofTDSI #DSSS@UofT. Raji's research focuses on algorithmic auditing & 2021-12-07 18:07:08 RT @emilymbender: For anyone wondering what this was about, it was (partly) in reference to what is now Raji et al 2021 in the #NeurIPS2021… 2021-12-06 21:45:38 @rachelmetz LOOL I wish I had the artistic skill! 2021-12-06 18:09:41 @DrDesmondPatton @SAFElab Congrats! They did the right thing. 2021-12-06 18:01:35 @Abebab CONGRATS! 2021-12-06 10:56:55 @benno_krojer But we don't currently have the structure to set this up as a research practice & 2021-12-06 10:54:10 @benno_krojer I do! A/B testing, functional tests, pilots, etc. are definitely all adequate alternative evaluation methods - even those technically without a human in the loop (think of sourcing live examples from customer interactions or an image search engine, etc.). 2021-12-06 01:50:16 @Abebab Thank you, Abeba!! 2021-12-05 13:19:31 RT @emilymbender: #NeurIPS2021 Datasets and Benchmark papers are now up in the preproceedings! https://t.co/12PQGvCdVX Thanks @joavansch… 2021-12-05 13:18:38 @joavanschoren @emilymbender @syeung10 @MariaXenoch @NeurIPSConf Thanks in advance for the help! 2021-12-05 13:18:26 @joavanschoren @emilymbender @syeung10 @MariaXenoch @NeurIPSConf Hey Joaquin, there happens to be a small formatting error in our abstract specifically, not sure if this can be fixed? https://t.co/wYCSC6mI9O 2021-12-05 01:11:47 @NeurIPSConf @mrtz And the fact that there's even a place for this work at NeurIPS this year (ie. a Datasets & cc: lovely collaborators @emilymbender, @cephaloponderer, @amandalynneP & 2021-12-05 01:05:21 This is a position paper - and thus just the beginning of what we hope will be a broader ongoing discussion on the role of data benchmarking in ML. Interestingly, there will also be a panel on this very topic @NeurIPSConf this year, moderated by @mrtz: https://t.co/TrU8qoIoWb https://t.co/vlqi4hXB7F 2021-12-05 00:58:40 To measure progress on models with broader capabilities, a single benchmark is not enough. Either use benchmarks for what they were originally designed for, to assess concrete progress on grounded applications & 2021-12-05 00:57:19 In our upcoming paper, we use a children's picture book to explain how bizarre it is that ML researchers claim to measure "general" model capabilities with *data* benchmarks - artifacts that are inherently specific, contextualized and finite. Deets here: https://t.co/hMqXsyuU1Z https://t.co/cIbUrR1gnd 2021-12-04 17:14:55 @reuben_aronson Yes. Any perceived short term benefit of overhyping performance claims is not worth the long term damage. It's not just about being caught in a lie (one that really hurts people!), it's also actually interfering with our ability to be honest with ourselves and tackle real work. 2021-12-04 17:04:44 AI hype is not just about caricature or a joke - unchecked it leads to premature, shoddy deployments, and causes real harm to real people. Lately, I get genuinely disappointed and upset when I spot it - in policy, or even marketing material. It's just not funny to me anymore. 2021-12-04 16:47:14 @mutalenkonde @easears @timnitGebru 2021-12-04 13:44:08 @percyliang @jennwvaughan And of course, major thanks to @IasonGabriel for blazing the trail with this work! 2021-12-04 13:43:20 @marylgray Thanks so much for your pioneering work in bringing these considerations into ML - we're all still just building & Also now that this & 2021-12-04 01:17:23 RT @michaelzimmer: I was among the 100+ (!!) ethics reviewers for @NeurIPSConf. Here's a summary of how that played out. 2021-12-04 01:17:14 @jennwvaughan Thanks for supporting this process 2021-12-04 01:16:40 RT @jennwvaughan: It’s been such a rewarding experience working with the amazing @rajiinio and Samy Bengio on this! We have a long way to… 2021-12-04 01:16:27 Next Fri, I'll be moderating a panel to discuss this further with experts @AmandaAskell, @Abebab, @JesseDodge, @cfiesler, @pascalefung, @hannawallach! Details here (+ check out the other amazing panels happening - moderated by @mrtz & 2021-12-04 01:16:26 It was such a joy to serve as Ethics Review co-chair with Samy Bengio @NeurIPSConf. The scale was ridiculous - over 100 ethical reviewers, over 450 ethical reviews - but beyond worth it to see authors, technical reviewers & 2021-12-03 23:30:30 RT @NeurIPSConf: Learn more about the #NeurIPS2021 ethics review process, including highlights and lessons learned, in this retrospective b… 2021-12-03 22:28:31 The most tragic thing is when people miss the deadline, because they forget to register their paper abstracts. DO NOT FORGET TO REGISTER YOUR PAPER ABSTRACT for @FAccTConference ‼‼ https://t.co/G95lrzfHjv 2021-12-03 22:26:24 RT @FAccTConference: SUBMISSIONS FOR ABSTRACT REGISTRATION IS NOW OPEN! A reminder that paper abstracts must be submitted before Dec 1… 2021-12-03 16:43:10 @mutalenkonde @timnitGebru @easears lol the revolution starts with Eric! 2021-12-03 16:30:18 @mutalenkonde @timnitGebru @easears Same here! @easears has been an incredible encouragement & 2021-12-03 16:17:00 Best thing about this article is the very intentional language of "AI marketed", "presented as AI", etc AI is not a real thing & 2021-12-03 15:21:59 @sedyst @AggieBalayn @sapiezynski Of course 2021-12-03 13:42:24 RT @sarahookr: Someone who regularly puts the spotlight on others is @hardmaru. I really appreciate all the small ways (often behind the sc… 2021-12-03 13:15:53 I keep returning to this report, crafted by @sedyst & Broadening the narrow discussion on "bias" to a more holistic conversation on "harms" is such an essential shift in the framing of AI policy. https://t.co/bb6ruNFJKb 2021-12-03 12:44:31 RT @ArlanWasHere: We don’t deserve @jovialjoy and @timnitGebru https://t.co/2876eF6c6g 2021-12-03 12:32:42 RT @_w0bb1t_: Fake #AI · From predicting crminality to sexual orientation, fake and deeply flawed AI is rampant .. Edited by @F_Kaltheuner… 2021-12-02 23:47:15 RT @KLdivergence: Almost six years ago, @wsisaac wrote an article using real data that showed how PredPol could exacerbate racial dispariti… 2021-12-02 19:23:44 RT @JuliaAngwin: Critics have long suspected that predictive policing software was racially biased. Today, we have the answer: @themarkup… 2021-12-02 19:23:33 RT @nealpatwari: Wow, congrats to this team for an amazing look behind the curtain! It's ironic that 6 years ago @KLdivergence, @wsisaac… 2021-12-02 19:03:24 RT @kharijohnson: Audits and assessments are being adopted by governments anxious to regulate AI and prevent discrimination, but there's li… 2021-12-02 15:41:59 I so much admire Timnit's courage, & 2021-12-01 21:08:10 RT @F_Kaltheuner: A strange time to launch something with the pandemic (rightly!) taking up all attention. Still, I'm very excited to shar… 2021-12-01 18:18:42 RT @lilianedwards: "We are not heading towards Artificial General Intelligence (AGI). We are not locked in an AI race that can only be won… 2021-12-01 14:17:14 RT @danyoel: Independent research shows speech recognition systems perform worse for AAVE speakers. New research reveals user frustration a… 2021-12-01 06:20:10 @swabhz @KrishnaPillutla @rown @jwthickstun @wellecks @YejinChoinka Congrats Swabha!!! Well deserved 2021-11-30 23:59:46 I'm so glad to see this paper rewarded - one of my favorites from the conference and such an important message! https://t.co/3AcVW8O5XZ 2021-11-30 23:55:26 @KLdivergence I dare you to write an intro with the word "decreasingly" lol 2021-11-30 23:54:36 @alexhanna @cephaloponderer Whoo! Congrats! 2021-11-30 23:53:33 RT @alexhanna: Wow! Honored that our paper with Bernie Koch, @cephaloponderer, and Jacob Foster won a best paper award at the NeurIPS Datas… 2021-11-29 18:08:38 RT @LeonYin: Trouble spotting Amazon private label products? @TheMarkup's new browser extension Amazon Brand Detector finds and highli… 2021-11-29 16:02:50 Imagine working for the @ACLU as an "Algorithmic Justice Specialist" Please apply! https://t.co/7YnA8es24r 2021-11-29 16:01:49 RT @Aaron_Horowitz: We have 4 open positions on the ACLU analytics team right now, all brand new frontiers for our team based on years of e… 2021-11-29 13:14:26 RT @anyabelz: #HumEval2022 deadline is 28 Feb - 3 months to get your paper ready for Second Workshop on Human Evaluation of NLP Systems at… 2021-11-29 01:54:33 @timnitGebru Still so in awe of this. So happy for you @IfeomaOzoma - you did not deserve anything that happened to you but so grateful you decided to invest your energy into making the situation more equitable for everyone. 2021-11-29 01:52:20 RT @veenadubal: I couldn’t agree more: “It’s remarkable how Ifeoma has taken some very painful experiences, developed solutions for them & 2021-11-28 23:27:24 RT @macfound: We're excited to support @ssrc_org's #JustTech Fellowship, just announced this month. This full-time $100K+/year fellowsh… 2021-11-27 03:41:17 @BlancheMinerva @RishiBommasani @PreetumNakkiran @davidwromero Yep this is a solid point & + you made a great earlier point that CLIP/DALLE is not necessarily the same as gtp-x and we should be careful what characteristics we use to set boundaries. 2021-11-27 03:36:50 @BlancheMinerva @RishiBommasani @PreetumNakkiran @davidwromero Did we need a new word? Probably not - we could have described the scenario with the set of qualifiers we already have (ie. Large pre-trained base models). But I do think raising awareness of the fact that this scenario happens so often & 2021-11-27 03:33:00 @BlancheMinerva @RishiBommasani @PreetumNakkiran @davidwromero My understanding is this - not all pre-trained models are base models for transfer learning & 2021-11-26 19:42:23 @emilymbender @MelMitchell1 + coming up on arxiv soon 2021-11-26 19:17:13 This was the most fun paper to write. I'm so happy we could leverage this analogy all the way to the end, aha. https://t.co/ffGLfB3Yby 2021-11-26 16:30:51 RT @MelMitchell1: This is such a perfect metaphor for AI. (From https://t.co/ZFaaO3cuL3) https://t.co/9vNzSiPOtY 2021-11-26 02:46:50 Petra Molnar, who wrote a @citizenlab report on the topic, captures why this is unsettling: “Decisions in the immigration context have lifelong and life-altering ramifications. People have the right to know ...so that we can meaningfully challenge.” https://t.co/QkvoCkzM1l 2021-11-26 02:46:49 This one hits a bit too... close to home. My uncle's rejected visa application emails look suspiciously like the templated answers spit out from this algorithm. https://t.co/pRmrhMIicA 2021-11-26 02:13:01 @joavanschoren @emilymbender @amandalynneP @cephaloponderer @alexhanna Thanks for your work advocating for and coordinating this track, Joaquin - I really appreciate that effort and think it will add a lot to this year's conference (+ hopefully future conferences as well)! 2021-11-24 19:33:21 RT @StanfordHAI: At this year's fall conference, scholar @rajiinio discussed her proposal for enabling and supporting third-party auditor a… 2021-11-23 20:25:51 @schock 2021-11-23 18:47:35 RT @qi2peng2: Crowdsourcing is hard, technical work. I wish one day I get to write a paper on a crowdsourced dataset that mainly focuses on… 2021-11-23 14:48:43 RT @emilymbender: @rajiinio @amandalynneP @cephaloponderer @alexhanna Research designing & 2021-11-23 06:53:55 @aselbst Ohh this is an interesting framing of the problem. Yeah, now that I think about it, it's pretty alarming that impacted non-users are so absent from HCI work. 2021-11-23 04:21:27 Was just reminded that this the term for what's happening. Such a good read by @mona_sloane, @MannyMoss, & https://t.co/54J84rVAmd 2021-11-23 01:56:28 I keep noticing that when companies want to show that they've consulted stakeholders to increase "participation", they will always choose users over the impacted population, even when the interests of these groups are clearly misaligned. A very real and recurring frustration. https://t.co/8XzbvLOEM4 2021-11-22 19:56:24 @ambaonadventure @mer__edith @sarahbmyers @oliviersylvain DREAM TEAM So thrilled for you all! 2021-11-22 19:54:48 There's finally a place at @NeurIPSConf for discussions on data and evaluation. It's worth everyone's time to check this out! https://t.co/6ylmZbSRus 2021-11-22 19:53:44 Daniel has done such impactful work with Algorithm Tips (https://t.co/YxUTV6wMIE) - excited to see what he does next! https://t.co/wOCA7S29sj 2021-11-22 02:36:42 RT @omertene: Dream job alert! The @OECD is hiring for Head of Unit – Data Governance and Privacy. This is a phenomenal tech policy/diploma… 2021-11-22 02:28:49 RT @rzshokri: What is the standard way of auditing data privacy for machine learning models? We have designed strong membership inference a… 2021-11-20 17:33:42 @WriteArthur oh interesting. Thanks for flagging, I'll take a look - I wasn't aware of this! 2021-11-19 21:06:26 @Adewunmi Yes, there's a paper I'm working on right now - will be able to share publicly soon! For now, feel free to cite the blog and/or presentation directly :) 2021-11-17 06:46:32 RT @jennwvaughan: Mark your calendar now!! On top of eight keynotes, #NeurIPS2021 will feature three plenary panels: - The Consequences of… 2021-11-15 21:57:39 RT @FAccTConference: We are committed to ensuring that no-one will be unable to present their work at the in-person conference due to re… 2021-11-15 15:01:10 I was fortunate to act as a technical advisor for this report & So excited to see as these ideas of third party auditor access evolve into practical realities - we need to support those already doing the work of holding platforms accountable. https://t.co/lwFrhgoEnm 2021-11-15 14:54:08 RT @drewharwell: Enjoying this dataset of thousands of images that often confuse AI systems, known as "natural adversarial objects." Donut… 2021-11-15 14:39:52 @brandonsilverm @vivian also doesn't work for me! 2021-11-15 04:48:13 Something cannot be more valuable or true because it uses a certain method, qualitative or quantitative. All these approaches have their limitations & 2021-11-15 04:39:13 I understand that we must emphasize the need for more qualitative work in a field that leans naturally towards valuing quantitative work but I also genuinely worry we don't think enough about when such methods are helpful or think carefully enough about how to properly execute. 2021-11-15 04:35:09 It confuses me when some valorize certain methods over others. Of course, quantitative methods aren't objectively "better" than qualitative methods - but the inverse is also true (ie social science can be just as abstract & 2021-11-13 01:07:22 In this paper, led by @amandalynneP, and w/ @alexhanna @cephaloponderer & 2021-11-13 01:07:21 We just published an extended version of the Data & Data has always been a critical aspect of machine learning but remains overlooked, under-considered and extensively mishandled in practice and pretty much ignored in theory. Why? https://t.co/gBxG7Riraw 2021-11-13 00:12:59 RT @amandalynneP: The new and improved version of "Data and its (dis)contents" is published at @Patterns_CP today! Co-authored with @rajiin… 2021-11-11 20:15:45 RT @dinabass: Facebook's facial recognition call doesn't mean most companies are backing off the tech, but it gives researchers & 2021-11-11 14:57:51 RT @haldaume3: agree with everything here i’d also point people to https://t.co/dcA5mqmY0T “AI and the Everything in the Whole Wide World… 2021-11-10 22:15:10 RT @WHOSTP: .@WHOSTP announces public events to engage the American public in national policymaking about AI and equity. Mark your calend… 2021-11-10 22:13:11 @SimoDragicevic @StanfordHAI @dpatil @mathbabedotorg @ProfFionasm @AvilaGarcez @chris_percy Thanks for sharing - I'll check this out! 2021-11-10 21:00:01 RT @StanfordHAI: The final session of the 2021 fall conference #RadicalPolicies4AI examined various perspectives on algorithmic auditing wi… 2021-11-10 17:37:42 @lauren_marietta Good question! @DanHo1, any idea? 2021-11-10 17:09:10 RT @FAccTConference: BIG NEWS! THE FACCT DEADLINE HAS BEEN EXTENDED! New dates: Abstract submission: 15 December 2021 Paper submis… 2021-11-10 17:02:16 Giving this talk today! Excited to engage in discussion about what it means to get third party participation to happen for algorithmic systems. I've had a lot of thoughts about this for a long time - glad to finally to present some of these ideas. https://t.co/sYRtMuC9Ma 2021-11-10 16:49:52 RT @drfeifei: Second day of @StanfordHAI Fall Conf - 2 more radical proposals for #AI and Policy, one on data cooperatives by @divyasiddart… 2021-11-09 19:20:10 RT @DrDesmondPatton: NEW OPPORTUNITY: @ssrc_org has just launched the #JustTech Fellowship, a 2-year, full-time fellowship offering $100K+/… 2021-11-07 06:14:11 @Abebab We just wrote this - about how weird it is that people keep trying to benchmark "general" AI capabilities, with data that is inherently subjective, scoped and limited lol https://t.co/ELtmdVZXM8 2021-11-06 23:20:00 CAFIAC FIX 2021-11-01 19:20:00 CAFIAC FIX 2021-11-01 17:30:00 CAFIAC FIX 2021-08-19 15:12:22 RT @nixonron: Today we begin publishing a series of stories exploring the impact of algorithms and AI on our lives every day. The first sto… 2021-08-18 22:50:19 @pranesh Of course there can always be some imagined benefits but there are many examples where that benefit even in the most idealistic scenario seriously dwarfs the cost (think of arguments against nuclear and automated weapons). We assume fundamental models dont fall into this category 2021-08-18 22:46:42 @pranesh Don't think that's true - more often, tech is inappropriately presented as "dual use" even when, due to issues with functionality or a hyper focus on a handful of harmful applications, the risk far outweighs benefits. I think this is the overly optimistic view she warns against. 2021-08-18 04:29:15 RT @caparsons: Apple says researchers can vet its child safety features. It’s suing a startup that does just that. https://t.co/ocK3lrVOnn 2021-08-16 18:09:13 @Aaron_Horowitz @ziebrah @hiddenmarkov @red_abebe @maxkasy lol yeah unfortunately they are often optimizing for just the volume and availability of the data, not its quality 2021-08-16 17:52:30 @ziebrah @hiddenmarkov Yeah, kind of reminds me of @red_abebe & Anyone in power can just decide on what it means to be worthy of a particular outcome...even if it's based on illogical assessments. Makes fairness impossible. 2021-08-16 17:35:26 @DataVizzdom @hiddenmarkov Yeah - it's here: https://t.co/NLFtwlbNQR 2021-08-16 17:29:15 @hiddenmarkov Yet another reminder that the pseudo-scientific tasks in ML go way beyond those obviously bogus facial recognition tools - we've already normalized making claims for ML to predict personality, emotion, criminality, etc. with inappropriate data from social media clicks, text, etc. 2021-08-16 17:12:39 So @hiddenmarkov just spotted the text based version of those phrenology/physiognomy papers... I mean, what does it mean to have the writing style of an “extrovert and liar”, and how would ML even learn that, and why should this be used to assess someone's loan application? https://t.co/eEZs6Okdgl 2021-08-15 15:33:30 @CatalinaGoanta @RebekahKTromble Thanks, will for sure take a look! 2021-08-15 14:16:48 @CatalinaGoanta @RebekahKTromble Agreed! Though how the DSA discusses the concept of auditing differs imo from how that activity is characterized in other policy contexts and how auditors will refer to themselves. These definitional differences are certainly worth addressing though - thanks for bringing this up! 2021-08-15 13:58:41 @RebekahKTromble Thanks for sharing this - will read further! 2021-08-15 13:57:29 @CatalinaGoanta @RebekahKTromble I don't think the term auditing needs to be used for the regulation to impact audit activity & 2021-08-15 13:54:16 @CatalinaGoanta @RebekahKTromble The "independent" auditors mentioned in Article 28 are not third party / external auditors - they are hired by the company and audit target. Functionally, Art 31 is the only one that seems to allow 3rd party auditors (in this case, just academic researchers) protected access. 2021-08-15 06:39:29 @RebekahKTromble Interesting - do you mind mentioning some examples? My understanding is that functionally, Article 31 is the most explicit instance of protecting third party auditors in the DSA, though you make a good point that it probably wasn't drafted with that intent. 2021-08-14 23:44:19 @natematias Wow this is an incredibly interesting analogy - thanks for sharing! 2021-08-14 15:54:35 @CatalinaGoanta Hm, in my mind there is a big difference between flagging bad content and getting protected access in order to identify more systematic issues in the moderation system. The latter is the role of the external auditor and to narrow that role to just academics is I think a mistake. 2021-08-13 16:37:22 RT @algorithmwatch: 1/9 AlgorithmWatch was forced to shut down its #Instagram monitoring project after threats from #Facebook! Read… 2021-08-13 16:24:52 A reminder that it's not just academic researchers that operate as third party external auditors to AI products. Investigative journalists, civil society, regulators, law firms and so many others can also play this role - and *all* of us need to be legally protected & 2021-08-13 15:31:47 Horrible - the way Facebook is strategically silencing third-party external auditors should be raising serious red flags for everyone hoping to see these companies being held accountable. https://t.co/MExtLHr8Kb 2021-08-13 15:09:30 Not enough people cite @LatanyaSweeney's early audit work in this space - glad to see that being mentioned here! She's a pioneer of the field and deserves more recognition for her contribution. https://t.co/bnU9l21tHM 2021-08-13 12:24:36 RT @daphneleprince: I wrote about this before when I spoke to @rajiinio, who is doing fantastic work to develop a methodology for algorithm… 2021-08-10 07:20:36 RT @EthanZ: Please support us in calling attention to this important battle between Facebook and NYU researchers. https://t.co/usK1WX3Fq0 2021-08-09 17:22:15 @KLdivergence Either way, totally happy for you you deserve fulfilling work in a safe environment.. and rest lol 2021-08-09 17:20:08 @KLdivergence lol Kristian I can tell you're really loving your new job 2021-08-09 15:41:50 Especially in machine learning, there is no real separation of "industry" and "research" - claiming this as a reason to abandon ethical responsibility as a researcher is such a weak excuse. It's no longer acceptable to say, "we don't need to think about this until deployment". https://t.co/ZcxruM8OCu 2021-08-09 15:28:30 RT @random_walker: To better understand the ethics of machine learning datasets, we picked three controversial face recognition / person re… 2021-08-09 15:14:03 RT @hiddenmarkov: Wow, this was an unexpected conclusion of the week! My submission was awarded the 1st place in the Twitter's Algorithmic… 2021-08-09 05:36:46 @hiddenmarkov @ruchowdh @TwitterEng Congrats Bogdan! 2021-08-07 21:37:32 RT @imchristiepitts: Me, at the airport, impulsively buying Wired bc @timnitGebru is on the cover https://t.co/saGSnGDWHc 2021-08-07 18:50:41 @emilymbender @jovialjoy @voguemagazine I know right?! Peak science communication lol and done in style 2021-08-07 18:46:49 Thrilled to see @jovialjoy in @voguemagazine! And challenging the biased image search results for "beautiful skin" We've gone mainstream, people! https://t.co/fCugR7IlOo 2021-08-07 18:38:11 @AllDeepLearning @alexhanna @Jenny_L_Davis @AprylW @safiyanoble Here you go: https://t.co/df59N8SqfF 2021-08-07 16:36:16 @alexhanna @Jenny_L_Davis @AprylW Wow, so excited to read this paper! @safiyanoble casually mentioned reparations in a panel once and it's been at the back of my mind ever since. Such a powerful framing! 2021-08-07 13:23:00 @MarkSendak @DrLukeOR @LukasBrausch @marylgray Just adding a note here to clarify that I've yet to comment on any of this yet & Though I suspect you may be talking past each other - Twitter isn't ideal for debate. 2021-08-06 18:49:42 RT @LeonDerczynski: "Double Standards in Social Media Content Moderation": new Brennan Center report: https://t.co/bNRSbRZDKC -- thanks @ra… 2021-08-06 14:29:08 RT @AbusiveLangWS: In 8 minutes, we open the 5th WOAH! Join us for a full day of keynotes from @LeonDerczynski Murali Shanmugavelan & 2021-08-06 00:47:21 This is why we need regulators to protect qualified third-party auditor access. Big tech companies should not have the ability to block external scrutiny of their products like this! https://t.co/upCkxrNzRM 2021-08-06 00:18:54 RT @anjalie_f: @rajiinio a lot times issues in AI/NLP applications are identified by communities of color, but the broader public / resear… 2021-08-05 22:09:04 RT @ZhijingJin: [Happening Today9am ET] We‘re hosting the 1st NLP for Positive Impact Workshop @NLP4PosImpact at #ACL2021NLP @aclmeeting… 2021-08-05 22:08:52 RT @MaartenSap: Also, I'll be moderating a panel on "The Future of NLP 4 Positive Impact" which will feature @pascalefung , @rajiinio, @bao… 2021-08-03 07:57:40 @wsisaac @DeepMind its really the least they can do after the revolutionizing you've accomplished congrats! 2021-07-30 18:00:58 RT @AfogBerkeley: In our first year, @AfogBerkeley member @AmitElazari championed the idea of bug bounties for identifying algorithmic bia… 2021-07-30 16:56:15 RT @TwitterEng: Calling all bounty hunters - it’s officially go time! We’ve just released the full details of our algorithmic bias bounty c… 2021-07-30 16:47:43 RT @geminiimatt: Much love admiration to the CRASH Project for trailblazing this research & 2021-07-30 16:23:50 RT @schock: @ruchowdh This is great to hear! Over at @AJLUnited we've got some related news to share https://t.co/EcmHHAgclV 2021-07-30 15:01:29 RT @AJLUnited: Happy Hacker Summer Camp Season! A CRASH Project update, from the team at @AJLUnited. https://t.co/s4RcStRtli https://t.co… 2021-07-29 22:26:35 RT @DrDesmondPatton: Excited to announce the Fall 21 lineup for our Race and Data Science Lecture Series at @DataSciColumbia. Please share… 2021-07-29 14:36:20 RT @jovialjoy: Your actions have inspired me to set boundaries I thought were not possible because of the weight of expectations. Too often… 2021-07-27 00:35:43 RT @schock: Two Muslims walk into a ... Large Language Model. https://t.co/J9dwmXVkTk 2021-07-26 20:21:39 @seanmmcdonald @jennifercobbe Thanks! 2021-07-26 20:18:07 @jennifercobbe Oh my. Very scared to ask but could you please elaborate? 2021-07-26 20:08:52 @mmitchell_ai @andrewthesmart @undersequoias @alexhanna Yet another thing we have in common 2021-07-26 19:18:27 RT @conitzer: Please share and respond to (by Sep 1) this request for public input on a National (US) AI Research Resource 2021-07-26 19:15:04 @Aaron_Horowitz @WHOSTP @Twitter Makes sense. Two of the most influential American institutions lol 2021-07-26 14:57:21 WOW THIS IS EXCITING! Congrats! https://t.co/J2x58woaLU 2021-07-25 02:26:57 @Miles_Brundage I don't think you'd personally call anyone a "hater", but this is language I see repeated explicitly by tech leaders following criticism of a product launch or AI tool. They say that to dismiss the criticism, by framing it as related to "other" things beyond their responsibility. 2021-07-25 02:24:11 @Miles_Brundage Yeah, sure - when it's implied that the criticism originates from motivations outside of pointing out failures/limits (and thus the harm these companies are responsible for), then it can be easily dismissed, related to things companies don't actually feel responsibility for. 2021-07-25 02:16:01 @Miles_Brundage Or they want things to work differently from how they do now. My main point though, is that this has much less to do with personal mistrust or dislike for individuals and institutions than you imply in the thread. Of course, I can only speak for myself and colleagues though. 2021-07-25 02:14:01 @Miles_Brundage I think a perceived de-coupling of criticism from failure/limits is exactly what people use to prop up the "hater" framing. You're right that I don't agree - I think many critics are concerned for those impacted & 2021-07-25 01:52:39 It can be disappointing to build something & 2021-07-25 01:48:22 I criticize to protect those communities with the knowledge I have in whatever way I can. I don't criticize because I don't like tech people or don't value tech companies. I mean, I've worked at tech cos, and have a lot of respect for tech workers. This has never been about them. 2021-07-25 01:45:11 I became disillusioned by the evidence. As in, I learnt more about how the technology worked and realized the narrative being told by these tech leaders was often misleading and most importantly harmful for communities I see every day and care about a lot. 2021-07-25 01:42:59 It is upsetting to see tech leaders respond to critics as though they are "haters", as if this is personal and there is some perceived value to negative reactions, because for many of us this is not the case. I came into tech optimistic and ready to believe in progress. 2021-07-25 01:38:57 I studied engineering & 2021-07-25 01:36:43 I catch glimpse of this perspective often - it genuinely surprises me. AI critics are not haters reluctant to give tech ppl their due credit. This in fact has nothing to do with tech ppl, is not at all personal. This is about those impacted & 2021-07-24 16:59:47 RT @sherrying: What’s your favorite band? https://t.co/Lbrxm9ukAx 2021-07-24 16:58:51 @alexhanna @sherrying @andrewthesmart Yeah - he told us! I feel like I got a limited edition original lol could be worth a lot in an NFT someday 2021-07-24 16:56:23 @sherrying @andrewthesmart ANDREW START SELLING THESE SHIRTS NOW 2021-07-24 15:53:49 @schock of course this would happen to you! Your twitter threads are legendary aha 2021-07-24 15:52:42 @ncooper57 Yeah, totally agreed with this - tests could have been more systematic. Thanks for sharing that paper, it looks great! 2021-07-24 03:41:18 @databoydg Congrats! 2021-07-24 03:26:25 Or any generated text for that matter (ie. to judge "this generated text is functionally doing what it's supposed to be doing - without mistakes") Code generation is really a brilliant use case for this! 2021-07-24 03:17:46 Like, imagine if we could measure the quality of machine translation based on operational functionality with some scenario test cases, as we can with code. Guess the closest we have right now is a human judge saying "yes, this translation is doing what it's supposed to be doing". 2021-07-24 03:14:57 I think about this a lot: "BLEU score may not be a reliable indicator of functional correctness. Functionally inequivalent programs generated by our model (which are guaranteed to disagree w the reference solution) often have higher BLEU scores than functionally equivalent ones." 2021-07-24 03:10:43 Don't think enough people talked about how interesting the evaluation situation was for Codex. It's a language generation model tied to a practical use case & 2021-07-23 19:28:54 @andrewthesmart @sherrying I get complimented every single time I put it on ahaha 2021-07-23 01:21:53 @lucy3_li Oh nice! Yes, let me DM you 2021-07-23 00:52:33 @chrmanning @ariannabetti Yeah - and I don't think this phenomenon is unique to NLP. There's a plethora of judgement-based tasks (inclu. toxicity detection but really most classification tasks) where there is no valid ground truth and annotation pretty much amounts to polling for majority perspectives. 2021-07-22 21:47:13 @davegershgorn Congrats Dave! 2021-07-22 18:19:31 @chrmanning @ariannabetti This looks great - thanks for sharing! 2021-07-21 15:17:37 @alexhanna @luke_stark All our faves taking over the white house - I love it aha 2021-07-21 15:17:05 @luke_stark ‼ 2021-07-21 15:16:02 RT @_KarenHao: This @WSJ investigation is incredible. The team created 100 bots with different profiles (age, location, interests) to watch… 2021-07-21 04:32:24 I honestly could go on and on. Rashida has been tirelessly pushing for tangible accountability outcomes in the AI space for years - it gives me such joy to see her now put in a position to do even more (& Trust me, we've all won with this appointment! Well deserved 2021-07-21 04:32:23 5. Most importantly for me though - in conversation but also through her work, I've learnt so much from her about race. She sees racial dynamics with clarity & Ref: https://t.co/eQ0ZqiSDCz, https://t.co/oQJnnidUZ1 https://t.co/Xoxe4x5nEC 2021-07-21 04:32:22 4. I also particularly appreciated this recent output - a paper on sorting through policy discussions to land at a practical definition of "automated decision systems" (ADS), a concept policymakers regularly misunderstand & Ref: https://t.co/nOOApXLrzk https://t.co/glLnALBZyN 2021-07-21 04:32:21 3. There's also this *award-winning* work with @ambaonadventure, defining the risks of "suspect development systems" (SDS). It's a solid case for how databases and not just algorithms require regulatory oversight, as consequential surveillance tools. Ref: https://t.co/aTXGdGxm2l https://t.co/FbGEEvLpcg 2021-07-21 04:32:20 2. While @AINowInstitute, she led & Ref: missing?https://t.co/Zl7zC6tvmd https://t.co/hQegckXyTN 2021-07-21 04:32:19 1. In the "Dirty Data" paper, she perfectly highlights a situation where "ground truth" can't be trusted, revealing how police departments that "juke the stats" (ie. manipulate data) still feed that corrupt data into predictive policing systems. Yikes Ref:https://t.co/UTU3i7vL1J https://t.co/J3TEAhOE5B 2021-07-21 04:32:18 Amazing news! Rashida Richardson is one of my favorite scholars. Her work has been transformational in the AI accountability space - she's revolutionized how we talk about data & For those unfamiliar, here are some of her papers I learnt a lot from. (THREAD) https://t.co/zogLs9wJIC https://t.co/2nYaMd2Mxk 2021-07-20 19:25:26 RT @mer__edith: This is such a wonderful appointment and so richly deserved The inimitable #RashidaRichardson is joining @AlondraNel… 2021-07-20 18:52:48 @struthious @841io @RDBinns oh, it's not what I was thinking of but this is a nice paper - thanks for sharing! 2021-07-20 18:50:55 @RDBinns @841io lol I don't think its any of these, strangely enough (the paper I'm thinking of has "utility" right in the title). But I love your title much better tbh aha the spice girl reference is gold 2021-07-20 17:49:16 RT @ButterKaffee: @tdietterich @asayeed Ground Truth is an illusion. Not only in NLP 2021-07-20 17:02:53 This is a good point, and something I now think about a lot. Sometimes annotators fundamentally disagree because they have different worldviews that cause them to perceive the data differently -- and not because anyone is particularly wrong. (eg. see: https://t.co/C15VSPLV6L) https://t.co/BhpuixUmql 2021-07-20 16:08:07 @jacyanthis I think policymakers have been willing to present at ICML workshops in the past, and even including a non-ML researcher with policy experience would provide a much needed difference in perspective. I'm sure organizers did their best, but the debate framing here is disappointing. 2021-07-20 14:32:24 *whether, ugh 2021-07-20 14:29:06 Am I missing something or is there not a single regulator as part of this discussion? It's possible the debate question is just awkwardly worded but it will not be up to the ML community wether or not they get regulated. Hopefully someone else will be making that decision. https://t.co/ZLjbGilqOh 2021-07-20 00:56:57 h/t @davegershgorn 2021-07-20 00:56:37 oil is the new data lol https://t.co/rBMVG8Po29 2021-07-19 21:37:17 @baykenney @black_in_ai Thanks 2021-07-19 21:36:52 @jennwvaughan @black_in_ai Thanks Jenn! 2021-07-19 21:36:29 @ruha9 @black_in_ai @Berkeley_EECS @berkeley_ai Thanks, Ruha!! 2021-07-19 21:35:58 @ndiakopoulos @black_in_ai @Berkeley_EECS @berkeley_ai Lol no pressure but thank you 2021-07-19 21:12:32 @841io @RDBinns Interesting. I recall that there's a CHI paper on exactly this idea of better measuring utility for online interactions, but can't remember the title 2021-07-19 20:44:36 Not at all an exaggeration to say that I'm in research today (and will hopefully continue) becuase of the support of groups like @black_in_ai. For anyone volunteering for any affinity group, know that your work is important & Representation matters! https://t.co/N0QpAD3HRf 2021-07-19 04:54:11 RT @fborgesius: “Auditing machine learning algorithms”, A white paper for public auditors, by the Supreme Audit Institutions of Finland, Ge… 2021-07-16 20:44:28 @_alialkhatib lol parental supervision required, truly 2021-07-16 16:59:53 @Aaron_Horowitz 2021-07-16 16:55:53 There's a Hardware Lottery (https://t.co/fmOhAc6kMo) and now a Benchmark Lottery - what if all progress in ML is just a sleight of hand...? https://t.co/Ap91xyDDRe 2021-07-16 16:42:48 RT @mmitchell_ai: I did an interview with Venturebeat about AI Ethics, Corporate structures, and Inclusion. Also talk a bit abt the firing… 2021-07-16 00:07:42 RT @PhilipDawson: Mapping of existing #AI standards development efforts onto requirements of EU Artificial Intelligence Act... @EU_ScienceH… 2021-07-15 21:10:50 @Combsthepoet 2021-07-15 14:15:45 @_datamimi @RoyalStatSoc @turinginst @rajinio @timnitGebru @jovialjoy Congrats! :) 2021-07-15 04:36:01 RT @katherine1ee: Data duplication is serious business! 3% of documents in the large language dataset, C4, have near-duplicates. Dedupl… 2021-07-15 04:29:31 RT @FOX2News: Staffers barred her entry saying she was banned after her face was scanned - saying Lamya was involved in a brawl at the skat… 2021-07-15 04:28:42 @undersequoias @Roblox it is so precious to me that he uses the "@" to try to pre-empt mismoderation in his defense lol what a hero 2021-07-15 04:26:43 RT @Combsthepoet: A 14 year old Black girl in Livonia, MI was misidentified by facial recognition at a skating rink. She was accused of bei… 2021-07-14 05:03:53 RT @ProfFerguson: Proprietary algorithms should not be allowed into evidence, if they can’t be tested by experts. This shouldn’t be hard fo… 2021-07-10 16:17:36 @JordanBHarrod Happy Birthday 2021-07-09 15:14:05 I learnt a lot about this from @red_abebe's paper on "Narratives and Counternarratives on Data Sharing in Africa" (see: https://t.co/c9ADtLHJhA). https://t.co/fIVdXEjnSO 2021-07-09 04:41:32 RT @Miles_Brundage: Excited to finally share a paper on what a huge chunk of OpenAI has been working on lately: building a series of code g… 2021-07-09 02:45:30 @krvarshney 2021-07-09 02:39:57 @Mehtabkn Yeah, these are great points! I do still think it's an under-utilized legal strategy tho, and my hope is that impacted individuals can still sue in civil court or something if the company's claims are proven false and disingenuous. 2021-07-09 02:17:35 Thanks @ruchowdh, @mona_sloane, @MannyMoss for writing this: https://t.co/9eZ2FhiYAq https://t.co/2wv1NfZRj6 2021-07-09 02:17:34 AI hiring tools have proven to be an excellent example of this - I don't want to hear about the 4/5th rule or EEOC when the product is pseudoscience. https://t.co/tRjCDYZmgb 2021-07-09 01:35:36 RT @LeonYin: If anyone wants a list of hate and social justice phrases to run thru, say, TikTok... Here are several we built with civil rig… 2021-07-08 08:52:37 @d4br4 It's clear to me why the reaction here is different, even when the stakes really aren't. 2021-07-08 08:52:24 @d4br4 Yup, that's a really valid point, and apologies if my initial reaction seemed to discount that! Honestly, the reply was more directed to other comments I was seeing at the time, comparing the copilot situation to the handling of CC licensed photos in image datasets, etc. 2021-07-08 08:20:16 @d4br4 Nope, I don't think writers don't care - there's just a less active community monitoring this the way the open source community does w/ code licenses. I'm not saying other cases of copyright violations are less important, just noting that it's unsurprising it's code sparking this 2021-07-08 01:36:29 I keep seeing comparisons to Creative Commons licenses lol The attention to detail & 2021-05-21 13:35:18 RT @schock: Because using a technology known to systematically discriminate against people with darker skin as a checkpoint to access unemp… 2021-05-21 00:56:40 RT @RoxanaDaneshjou: And if you think that the FDA will protect us, please check out our paper on AI devices in medicine: https://t.co/BgrK… 2021-05-21 00:52:32 @RoxanaDaneshjou @alexhanna @Abebab Great thread! I especially appreciate the point on advertising performance despite the lack of clinical trials. One of my main frustrations with ML in healthcare applications. 2021-05-21 00:49:26 RT @RoxanaDaneshjou: What does it mean to launch a tool "meant to help patients" but shouldn't be used as a diagnostic? Google is launching… 2021-05-20 23:41:53 RT @jackbandy: For those who were interested, I put a template for the "dataset nutrition label" card on @overleaf! Should be easy to reuse… 2021-05-20 17:01:01 RT @zenalbatross: when asked about the risk of the app misdiagnosing Black and brown folks, Google's response was basically 'It's not our f… 2021-05-20 17:00:33 RT @zenalbatross: NEW: Google released an app that diagnoses skin conditions based on photos.... but it won't work on people with darker sk… 2021-05-19 22:55:21 @jefrankle @roydanroy @cfiesler @jovialjoy @60Minutes @AJLUnited Thanks - though we were definitely directly influenced by your early work at Georgetown! "Perpetual Lineup" was the first thing Joy made me read when I started working with @AJLUnited lol https://t.co/KWolYPpKXy 2021-05-19 22:00:39 @mnirPRJCTS Oh so sorry - I am not good at email and it's likely in spam depending on where you sent it! I'll dig up the email and respond shortly 2021-05-19 21:54:02 In light of frequent erasures, we might need something like this but for AI? https://t.co/DfGCbHUu2L 2021-05-19 21:00:25 RT @alvarombedoya: There is a "nothing-to-see-here" tone to this statement from @60Minutes that, combined with the failure to make any ment… 2021-05-19 20:44:53 @roydanroy @cfiesler @jovialjoy @60Minutes @AJLUnited I know it's unproductive to focus on "firsts" but Joy is indeed a pioneer here. Much of our knowledge of intersectional & 2021-05-19 20:39:50 @roydanroy @cfiesler @jovialjoy @60Minutes @AJLUnited The 60 minutes segment was about individuals misidentified because of racial disparities in performance, and the NIST study they focus on is the 2019 one, which was the first one in which NIST includes evaluations for race for the first time (all because of @jovialjoy's work)! 2021-05-19 20:36:28 @ghoshd @roydanroy @cfiesler @jovialjoy @60Minutes @AJLUnited Not necessarily - though I do see audits as being part of a participatory evaluation approach of sorts. I find them to be one mechanism for humans to collectively assess models and determine if it's appropriate for deployment in a given context. 2021-05-19 20:34:37 @roydanroy @cfiesler @jovialjoy @60Minutes @AJLUnited Also, before Gender Shades, no one was evaluating this performance intersectionally. Looking at performance disparities across subgroups at the intersection of race and gender, for example, reveals more remarkable failures than looking at disparities across unitary subgroups. 2021-05-19 20:32:15 @roydanroy @cfiesler @jovialjoy @60Minutes @AJLUnited But afaik, NIST didn't audit for performance differences due to race until 2019. This was was the first year NIST included a "part 3" for FRVT (for a demographics performance disparity test), and they cite Gender Shades as the direct inspiration and motivation for them doing so. 2021-05-19 20:28:50 @roydanroy @cfiesler @jovialjoy @60Minutes @AJLUnited There are other papers about a decade later referencing similar observations (eg. "Face Recognition Performance: Role of Demographic Information" from 2012 and "An other-race effect for face recognition algorithms" in 2011, which we regularly cite in our work). 2021-05-19 20:26:26 @roydanroy @cfiesler @jovialjoy @60Minutes @AJLUnited The 2002 NIST reference is probably from one of the first FRVT results, which includes this comment: "Demographic results show that males are easier to recognize than females and that older people are easier to recognize than younger people." https://t.co/0N4ZEzmq9d 2021-05-19 19:53:05 @ndiakopoulos @NeurIPSConf Happy to see this work! 2021-05-19 19:52:45 RT @priyakalot: Curious about how AI researchers are thinking about the societal consequences of their work? We (@JessicaHullman @ndiakopo… 2021-05-19 19:52:25 RT @ndiakopoulos: Hope the @NeurIPSConf organizers will take note https://t.co/9CoIImet4X 2021-05-19 13:50:40 @Combsthepoet @schock @imlwilliams @jovialjoy @timnitGebru Thank you for fighting so hard despite the struggles and lack of recognition. I know it comes from pure love 2021-05-19 13:41:56 Seriously. All this advocacy took so much energy & I am literally begging journalists to be more honest about this. These are unbelievable outcomes from large tech cos - name those involved in making this happen https://t.co/LFFCURTPxe 2021-05-19 13:25:55 RT @AJLUnited: This is great news!! When we rise up together, justice wins! #SurveillanceIsntSafety https://t.co/G59O3MR6Yg 2021-05-19 01:09:45 Wow - @timnitGebru @iajunwa & No doubt that some serious truth is about to be dropped. https://t.co/ZzrjselaV5 2021-05-19 01:06:32 RT @conitzer: The AI, Ethics, and Society Conference starts tomorrow (Wednesday)! @AIESConf See the program here: https://t.co/Lfpbukygl2 2021-05-19 01:00:22 RT @richardjnieva: New: I chatted with Google AI chief Jeff Dean about the controversy involving @timnitGebru and @mmitchell_ai. "The reput… 2021-05-18 20:40:46 @Leo___Sturm Yep - decisions are being made, we're just asking to make them more carefully and with intention, please. No mythically "unbiased" dataset exists, neither is it necessary here. 2021-05-18 20:31:10 Bias in data is inevitable but there's nothing inevitable about deploying a product that doesn't work on a vulnerable population. 2021-05-18 20:17:59 RT @Combsthepoet: I don't have the energy to say all I want to say, but I also spoke to 60 mins about the organizing on the ground in Detro… 2021-05-18 19:53:14 RT @ClareAngelyn: I did NOT expect this but am not at all mad about it https://t.co/lBPqeVMXY3 2021-05-17 14:59:29 @geomblog @WHOSTP @AlondraNelson46 @BrownUniversity @BrownCSDept @Brown_DSI @senykam @UtahSoC Congrats! Never been more excited to have someone join the government lol - please tell them everything, especially all that stuff about AUC being a useless metric 2021-05-17 14:11:06 By the way, if you're looking to see the absolute other side of the spectrum - a feature on algorithmic bias with Black people everywhere, highlighting the diverse pool of researchers working on this issue, then I really suggest checking out this recent episode of #UnitedShades https://t.co/Jm5ixtzVY5 2021-05-17 13:36:16 @karaswisher @jovialjoy the irony here is literally killing me 2021-05-17 13:12:46 @jovialjoy @AJLUnited This is the thing that frustrates me the most. How could someone waste your time like that? As if you don't have so many other things to do. It's unbelievable! 2021-05-17 12:38:32 @jovialjoy And this time, the choice was so deliberate - that NIST study cites all of our work directly as inspiration & 2021-05-17 12:29:55 I'm getting tired of this pattern. At this point, @jovialjoy has to spend almost as much time actively fighting erasure as she does just doing her work. It's a waste of everyone's energy & 2021-05-17 12:16:43 RT @jovialjoy: @60Minutes producers spoke to me for many hours. I even spent additional time building a custom demo for @andersoncooper and… 2021-05-16 20:14:14 @wsisaac I don't know if this is an actual pattern but I think a lot of social scientists read Rubin and a lot of ML folks read Pearl - it's funny to see you reading both! 2021-05-16 20:10:17 @wsisaac Omg why do I have two of these books myself 2021-05-14 20:36:25 RT @NeurIPSConf: The #NeurIPS2021 paper submission deadline has been extended by 48 hours. The new deadline is Friday, May 28 at 1pm PT (ab… 2021-05-14 11:17:14 RT @anyabelz: Very excited to announce that the #ReproHum project on Reproducibility of Human Evaluations in #NLProc with @ehudreiter is ge… 2021-05-13 20:06:17 RT @CohereAI: Model cards are an important step towards the responsible productization of machine learning and the usability of derived too… 2021-05-13 20:05:44 RT @paperswithcode: Introducing Datasets on arXiv! The new "Code & Read mo… 2021-05-11 13:04:28 @benwagne_r 2021-05-10 22:46:49 @Abebab Billionaires in general lol - they don't even have to give their money away. 2021-05-10 22:44:02 RT @eveikey: Please join us for @rajiinio's exciting talk on Wednesday at 4pm Pacific! The session is open to all, and ASL services will be… 2021-05-10 22:43:36 RT @DesignLabUCSD: This Wednesday's #DesignAtLarge, join us as Inioluwa Deborah Raji (@mozilla) sheds some light on the problems we face wi… 2021-05-09 23:13:38 @BlakeleyHPayne @cfiesler @allergyPhD @Greene_DM @morganklauss Thanks! @amironesei contributed greatly to that paper as well - it's here for anyone that may be interested in checking it out: https://t.co/R6iFUQFAqu 2021-05-07 13:55:06 RT @EugeneVinitsky: What if we just didn't bold the overlapping distributions https://t.co/XYtBGRbwso 2021-05-06 21:30:39 RT @SEDL_workshop: We have a great line-up of speakers and panelists: @alexhanna Joëlle Pineau @adinamwilliams @pushmeet @adyasha10 @rajiin… 2021-05-06 20:11:35 RT @marc_schulder: From a #ACL2021NLP reviewer perspective I must say I was very happy about the new ethics board and guidelines. It gave m… 2021-05-06 15:01:19 RT @bluevincent: Tomorrow I will serve as moderator for the Social Impact of ML Research session featuring Deb Raji (@rajiinio), Adina Will… 2021-05-06 14:49:33 RT @AdaLovelaceInst: Today we're excited to announce a new project exploring solutions to the unique ethical risks that are emerging in ass… 2021-05-05 23:18:39 @ZeerakW @AlvinGrissomII @Abebab Oh, that's fair - it's common to just use whatever info you have about people to gauge first impressions. I guess I'm thinking of those that go a step further to really shame people for taking a job they need. And these people are often hyper-focus on minorities :( 2021-05-05 22:58:44 @ZeerakW @Abebab oh I respectfully disagree here - I don't think rank & 2021-05-05 22:18:29 RT @sayashk: @rajiinio @Abebab Someone shared this excerpt here a while back (from Kai Cheng Thom's book), and it seems very applicable! ht… 2021-05-05 22:13:56 @sayashk @Abebab Amazing - thanks for sharing this! 2021-05-05 21:29:04 Also, it's unbelievable that someone is targeting @Abebab, of all people. She's more fearless in her critique of companies than anyone I know. Pls stop acting like corporate affiliation is the entirety of what a person is - declare conflict of interest when relevant & 2021-05-05 21:29:03 Why do people do this? If you don't believe in working at a company, then that's fine - but why police someone else's choice? ML/AI products are a veritable mess - there's more than enough work for those on the outside and lots of internal accountability work to be done as well https://t.co/gAGmo2e2jk 2021-05-05 14:56:40 @Gabuhamad @FAccTConference @AIESConf For my own sanity, I can't get involved in yet another project, but please share anything you write about this - more than happy to cite & 2021-05-05 14:55:37 @IasonGabriel Thanks for your leadership on this at Neurips! 2021-05-05 04:52:39 @timnitGebru @negar_rz @FAccTConference @AIESConf 2021-05-05 01:46:05 I'm pretty excited to see mainstream ML conferences like NeurIPS and ICLR incorporating some level of ethical reflection in reviews. This is a step that I don't think even @FAccTConference and @AIESConf have taken yet. https://t.co/OPLzsj3mjg https://t.co/1g4Ph1dZ2k 2021-05-04 22:27:53 RT @arxiv: arXiv is free to use, but not free to operate. If everyone who visits arXiv this month gives just one dollar, we would meet our… 2021-05-04 17:22:13 RT @shakir_za: The #ICLR2021 Town Hall is coming up soon. If you are participating, the OC update has so much good info on ethics, thoughts… 2021-05-04 16:52:04 RT @bsmith13: I'm hiring a Policy Research Analyst! https://t.co/4VQ4MdHQYq https://t.co/OYnjOQGU5F 2021-05-04 03:01:07 RT @peard33: Apple hires ex-Google AI scientist Samy Bengio who resigned after colleagues' firings https://t.co/lxkWqK6dHc 2021-05-04 02:59:01 RT @rishiyer: Hi @NeurIPSConf, sincere requests from many of my friends, collaborators (and most importantly students) affected by COVID to… 2021-04-30 16:56:29 RT @nowthisnews: A false match by racial recognition technology sent this man to jail for a crime he didn’t commit https://t.co/F2SOsHWCHR 2021-04-30 15:08:16 RT @SNolanCollins: During last Tuesday's hearing, @maziehirono asked Facebook if they were complying with all federal civil rights laws. Fa… 2021-04-30 12:18:19 RT @wsisaac: The NYPD is ending its controversial robot dog trial https://t.co/V6gZy5K8NF 2021-04-30 00:43:10 RT @math_rachel: Saving Face: Investigating the Ethical Concerns of Facial Recognition Auditing, from AIES 2020, by @rajiinio @timnitGebru… 2021-04-29 22:33:58 RT @hypervisible: “…because no federal guidelines exist to limit or standardize the use of facial recognition by law enforcement, states --… 2021-04-28 16:59:28 RT @GretchenMarina: @math_rachel @rajiinio @AINowInstitute I also like this piece by @alfredwkng in @themarkup (which itself might be on my… 2021-04-26 20:47:37 @EugeneVinitsky @NeurIPSConf @jennwvaughan Personally though, I agree that if an extension was provided to those affected by US protests last summer, one should definitely be made available to those meaningfully impacted by India's COVID situation. 2021-04-26 20:45:44 @EugeneVinitsky @NeurIPSConf I'm not sure - I feel like there should be a way to collectively contact program chairs directly to make such a request ? @jennwvaughan may have some feedback on this. 2021-04-25 21:41:19 Many things in this thread are true but I want to highlight this. I spent a long time focused on identifying a safe space for myself to do my work. Some tried to rush me & 2021-04-25 21:18:45 I agree - and we shouldn't hesitate to make such direct requests to @NeurIPSConf (or any conference) organizers, especially when this kind of accommodation is necessary. Sending much love and strength to those affected by the dire COVID situation in India right now. https://t.co/Q0EscWE7PL 2021-04-25 20:16:09 RT @hima_lakkaraju: As I struggled to deal with the impact of COVID on my family members in India, I got delayed by a day for submitting my… 2021-04-24 12:34:24 I can't reiterate how much this is true. Even AI critics will fall for the PR hype, discussing ethics in the context of some supposedly functional technology. But, often, there is no moral dilemma beyond the fact that something consequential was deployed & 2021-04-22 18:20:15 Love how much @radical_ai_ 's one brave question at the @FAccTConference 2019 Town Hall shifted people's thinking on this. She went up to the mic, quoted "Data Feminism" & 2021-04-22 14:06:37 @andrewhu_ @morganklauss @amironesei Thank you! Glad you enjoyed it :) 2021-04-22 14:03:05 @mmitchell_ai @timnitGebru @JeffDean 2021-04-20 23:05:34 RT @drlungamam: Police were alerted to the area that ended in the murder of 13 yo Adam Toledo by Shot Spotter, a smart city technology incr… 2021-04-20 19:56:36 @Combsthepoet I'm so nervous... my mind has been on this all week. 2021-04-20 19:40:27 RT @ComputerHistory: .@rajiinio's one word of advice is "courage", because it will take courage for the next generation to create change. "… 2021-04-20 19:37:00 RT @melindagates: I’m very grateful for @jovialjoy’s leadership in the growing movement to confront bias in algorithms. I hope as many peop… 2021-04-20 19:33:42 RT @ComputerHistory: "We treat racist AI as if they're a racist human being, not an automated technology. Usually, you have a situation whe… 2021-04-20 19:33:26 RT @ComputerHistory: "It's very easy to forget the people at the intersection. It's easy to forget Black women when talking about Black peo… 2021-04-20 19:33:18 RT @ComputerHistory: "Machine learning requires a lot of data. Having to define the datasets, label it, makes people concerned since it add… 2021-04-20 19:33:02 RT @ComputerHistory: "Institutional barriers are some of the hardest to resist, and that includes the ones in tech." - @rajiinio #CHMLive 2021-04-20 19:32:46 RT @ComputerHistory: "Sometimes these failures happen to groups that are most vulnerable. If you have a police department that's prejudiced… 2021-04-20 19:32:34 RT @ComputerHistory: "As someone who's part of the Black community, I am emotionally affected by seeing how people of color are impacted. W… 2021-04-20 19:32:25 RT @ComputerHistory: "I've felt this lack of response even in spaces that are meant to talk about AI ethics or fairness. Those concerns get… 2021-04-20 17:19:42 As an aside, here's a run-down of another creative audit @LeonYin designed - using a "staining" method to identify the amount of real estate Google was giving to its own products in the search result pages. Still one of my favourite things to read about. https://t.co/O5V75eivXG 2021-04-20 17:19:41 Wow, check out the methodology for the latest @themarkup audit of Youtube advertising blocklists. Journalists, regulators & https://t.co/84RDXVLTqb 2021-04-20 16:21:34 RT @jackbandy: Ever wondered how your algorithmic timeline is different from the old-fashioned chronological timeline? Lucky for you,… 2021-04-19 22:03:44 @swabhz @CSatUSC @nlp_usc Yay! 2021-04-19 18:25:40 RT @karaswisher: Important interview with ⁦@jovialjoy⁩ about where we are since she unmasked racial bias in facial software: Opinion | She’… 2021-04-19 03:59:24 And we absolutely need regulation, to address situations where corporate incentives are fundamentally misaligned with preventing harm. Advocating for internal accountability is not the same as advocating against external accountability. That's a false dichotomy - we need both! 2021-04-19 03:59:23 Projects like ABOUT ML is what an idealized PAI should be doing. If you ever talk to @mmitchell_ai, @hannawallach or @erichorvitz about the org, this goal is immediately clear. However, it's unclear to me if the field or the org itself understands these issues to be a priority. 2021-04-19 03:59:22 For example, in the face of several embarrassing failures, some companies (IBM, Google, Microsoft, etc.) realized engineers weren't even documenting what was happening & 2021-04-19 03:59:21 Ideally, interventions like @PartnershipAI are supposed to support industry in addressing situations within Case 1. In other fields where functionality is also a safety issue (medical devices, aerospace, etc), they have similar industry collectives thinking through best practices 2021-04-19 03:59:20 "Case 2" problems are not like this. To stop deploying a perfectly "functional" product, for ethical reasons alone, completely contradicts corporate incentives. We should be suspicious of company input on such issues & 2021-04-19 03:59:19 I often frame "Case 1" as engineering responsibility or safety issues - in my opinion, these situations should not be controversial. It's much more straightforward to refute stated performance claims than make a moral argument for why a functional product should be recalled. 2021-04-16 16:21:23 RT @ComputerHistory: As a computer scientist and activist, @rajiinio has worked closely with @AJLUnited on several award-winning projects t… 2021-04-16 16:20:33 RT @ComputerHistory: Is AI racist? Can it be used for anti-racist outcomes? Join us on 4/20 for a conversation featuring @lilich, @cmcilwai… 2021-04-16 13:05:27 RT @baxterkb: Look who are the keynote speakers at the @AIESConf (#AIethics & 2021-04-16 11:50:30 RT @autreche: Super excited to co-organize the ICML Workshop on Algorithmic Recourse (https://t.co/GKuGg52okP) with @hima_lakkaraju @strati… 2021-04-16 11:48:10 RT @bansalg_: Do you work on AI/ML + HCI? We invite submissions for our new journal--- Special Issue on "AI for (and by) the People" http… 2021-04-15 16:38:45 @reshamas @PartnershipAI @hannawallach @mmitchell_ai @OTI @EFF @IBM @Sony @LeverhulmeCFI @jennwvaughan @timnitGebru @Jinyingyang @HannahWallach I know there's this: https://t.co/8fXNhopi5n @jingyingyang can definitely point you to where the draft white paper now lies. 2021-04-15 12:11:23 RT @anyabelz: So, wow. We have 326 people registered for #HumEval workshop at #EACL2021!!! If anyone else wants to come, hey I'm sure we ca… 2021-04-15 10:48:40 @DMulliganUCB @luke_stark @Berkeley_EECS @beenwrekt @red_abebe @jennaburrell @AfogBerkeley @BerkeleyISchool @emma_lurie @JiSuYoo1 @zoebkahn @lizzielansdowne @xiao_sijia @liza_gak Nice - looking forward to meeting everyone!! 2021-04-15 10:43:05 @LeonYin With the initials it's going to be Dr. D.R. and I'd be lying if I said this doesn't make me just a lil excited. 2021-04-15 10:35:53 @lucy3_li @Berkeley_EECS @beenwrekt @red_abebe @BerkeleyISchool Definitely! 2021-04-14 21:42:13 @red_abebe @databoydg 2021-04-14 21:41:38 @red_abebe @Berkeley_EECS lol you really are the best at recruiting tho. 2021-04-14 21:21:21 As an aside: minority students will work with you if you value their work, make them feel safe & 2021-04-14 21:21:20 Like so many woc, my undergrad experience was difficult, socially isolating & 2021-04-14 21:21:19 I'm starting a CS PhD @Berkeley_EECS this Fall, working with @beenwrekt & Doing a PhD is a casual decision for some - that wasn't the case for me. I appreciate everyone that respected me enough to understand this & 2021-04-14 18:39:31 @mmitchell_ai I wish I could give a lecture simply titled "Erasure", and just list out one instance after the other of this happening to minorities and women in the ML field over and over, for no reason whatsoever. 2021-04-14 18:00:44 No one is allowed to forget the origin story of ABOUT ML (https://t.co/8fXNho7HdP). These women were pioneers in establishing documentation practice not just at their respective institutions (Google, Microsoft) but across the entire industry! No a small feat, at all. https://t.co/9k7MZ0GTEu 2021-04-14 17:45:06 @mmitchell_ai @PartnershipAI @OTI @EFF @IBM @Sony @LeverhulmeCFI @hannawallach Very strange to omit you two - who really pushed for the partnership to start this initiative in the first place 2021-04-14 10:58:55 RT @LeonYin: YouTube blocked advertisers from finding videos related to "Black Lives Matter," but not "White Lives Matter." https://t.co/M… 2021-04-13 20:14:52 RT @drewharwell: New: Detroit police wrongfully arrested Robert Williams in front of his two young daughters after a bad facial recognition… 2021-04-13 19:59:00 @yoavgo @timnitGebru @nsthorat @mmitchell_ai Sure, but this is research. It's not as if her participation was exchangeable, her contributions were very unique to her as a person. They didn't value what she did for them, which is why the dismissed her and did so disrespectfully. Either way, we can agree to disagree lol 2021-04-13 19:54:04 @yoavgo @timnitGebru @nsthorat @mmitchell_ai Yes, that's right. Why don't you think that's the case? 2021-04-13 19:52:03 @yoavgo @timnitGebru @nsthorat @mmitchell_ai I think Google values both diversity and ai ethics work in the abstract, but was dismissive of Timnit's specific contributions to these broader efforts in both cases. 2021-04-13 19:50:42 @yoavgo @timnitGebru @nsthorat @mmitchell_ai Yeah - it's possible there's some disagreement here & 2021-04-13 19:38:38 @yoavgo @timnitGebru @nsthorat @mmitchell_ai I don't doubt that ethics work continues to be valued at Google. I just don't think @timnitGebru's own legitimate contributions to that work was valued in the way it should have been. Her dismissal happened in a context where her specific contributions were truly being sidelined. 2021-04-13 19:33:41 @yoavgo @timnitGebru @nsthorat @mmitchell_ai Hm, I don't think that's quite right. I think the claim is that there's a version of ethics work that is valued at Google, but the valid contributions of especially marginalized individuals are not equally valued as part of that work. They are thus more easily dismissed. 2021-04-13 19:26:33 What Timnit says here is so correct in so many ways. The lack of diverse leadership at most AI ethics institutions reveals how much more can be done to protect & 2021-04-13 19:14:53 @timnitGebru @nsthorat @mmitchell_ai 2021-04-13 18:16:54 @timnitGebru @nsthorat @mmitchell_ai Yep - even if this general work could still be valued at the company, it's clear that your specific contribution to that broader agenda was not valued. I think this is my point. 2021-04-13 18:14:50 RT @D_schoolz: Im not special and not in the AI community but I stopped interviewing with Google last month for mobile software engineering… 2021-04-13 17:42:43 @nsthorat @timnitGebru @mmitchell_ai Happy to hear this but I think Google definitely underestimated their impact on the research community, for sure - otherwise the dismissals would have been handled with much more caution & 2021-04-13 17:15:03 I said this because I believe it: “It was really easy for them to fire her because they didn’t value the work she was doing”. Unfortunately for Google, much of the research community disagrees. Proud to continue standing by @timnitGebru & https://t.co/SDxjwXfkv9 2021-04-13 17:00:48 RT @jjvincent: The fallout from Google's firing of @timnitGebru and @mmitchell_ai continues to shake the AI community — I wrote about ongoi… 2021-04-12 20:03:33 RT @StanfordAILab: The Stanford AI Lab supports and deeply appreciates the talented Iranian members of our community. We strive for equitab… 2021-04-12 15:45:53 @DataSciBae But there's so much more though. I want to see romcoms! And the past has Black joy, if people just pay attention. I want to see pre-colonial African royal dramas 2021-04-12 15:08:18 RT @sapiezynski: This shouldn't be controversial but it apparently is. If you're doing something shown to be harmful, you should not get t… 2021-04-12 12:54:42 RT @ReviewAcl: ARR is now accepting submissions! Please see https://t.co/wiXNAvVhZO for an overview of the submission form and link to the… 2021-04-12 05:05:56 RT @techreview: The 10 most cited AI data sets are riddled with label errors, according to a new study out of MIT, and it’s distorting our… 2021-04-11 20:54:18 @agstrait @ruchowdh 2021-04-11 10:54:50 RT @black_in_ai: Appreciation post for @sindero who is a black pioneer in the field of machine learning (20y experience) and is the co-inve… 2021-04-11 01:20:29 Simon was the first real ML researcher I met (sat beside me at the first @black_in_ai workshop!) - so annoyed to hear of anyone attempting erasure of his many important contributions. https://t.co/5TM9hgzMCX 2021-04-09 21:04:43 RT @korolova: Facebook’s ad delivery algorithms have been known to introduce bias and create echo chambers for job, housing and politics ad… 2021-04-09 16:50:45 RT @_KarenHao: The latest audit of FB's ad service shows it's still excluding women from seeing job ads without regard to their qualificati… 2021-04-09 14:26:11 Shouldn't there be more serious consequences for this? The POST Act (which passed through NYC City Council this June) has provisions "requiring the NYPD to disclose basic information about the surveillance tools it uses" (see: https://t.co/LtWVRjtBNh). https://t.co/45f97YABuj 2021-04-09 00:29:35 @BobGoffer Agreed! 2021-04-09 00:27:46 @JulianPosada0 It's actually an evaluation dataset, so not technically meant to be used for training. But yes, this is a good point - there doesn't seem to be meta-data on geographic context. Do you think that would cause important changes to the image/video/audio? 2021-04-08 22:17:50 RT @VPjedwards: Breaking: It will be illegal for nearly all Virginia law enforcement agencies to use facial recognition technology starting… 2021-04-08 22:04:34 RT @hypervisible: "The models failed to detect faces in images labeled as including Black faces 57 percent of the time. Some of the failure… 2021-04-08 22:02:54 @BobGoffer Oh, thanks for sharing - I'll take a look! 2021-04-08 21:55:09 @BobGoffer Thanks for sharing that context. Yeah, it's worth thinking about how this will play out as a resource within the company. I was actually thinking of this as a resource for the audit community - given how difficult it can be to ethically source diverse data for evals, it's useful. 2021-04-08 20:13:11 RT @rajiinio: Also, to clarify just how *difficult* this problem of ethically constructing such datasets are - take a look at this paper on… 2021-04-08 19:13:23 @dlowd Yeah, I understand completely. My understanding is that companies this large also contain many great researchers doing good work (@mbogen, for example!) despite the visible mistakes made by leadership, policy teams, etc. 2021-04-08 19:07:04 @AutoArtMachine Yeah, I agree that would have been valuable here - I think the motivation for doing this was that for a computer vision system, which the dataset was created to evaluate, the "visual cue" determining prediction performance is skin type, not necessarily self-identified race. 2021-04-08 19:06:19 @hannawallach btws I really enjoyed this paper! We explored similar issues in our "Saving Face" paper (reflecting on the challenges of doing Gender Shades-style audits), you might like it! https://t.co/qyGrhn3hHF 2021-04-08 17:28:35 Also, to clarify just how *difficult* this problem of ethically constructing such datasets are - take a look at this paper on "Designing Disaggregated Evaluations of AI Systems". We also discuss similar challenges in "Saving Face"(https://t.co/qyGrhn3hHF). https://t.co/vjtMhd09GW 2021-04-08 17:21:15 h/t @lizjosullivan for putting this on my feed 2021-04-08 17:20:05 To be clear, facial recognition in particular is a technology I don't support - it hurts people when it doesn't work and also hurts people when it does. But as we push back, it's important to flag that current failures are inexcusable. You can't deploy a tool & 2021-04-08 17:12:52 They paid them and asked for consent. This is larger & Functionality is just one of *many* issues, of course, but now there's literally no excuse for a tool not to work on minorities. https://t.co/A6iyyjaiY5 2021-04-08 13:34:32 @kgajos We had a workshop featuring some meta-papers that included examples of this after @NeurIPSConf's requirement last year (deets at https://t.co/V8izkEO9Yo). I think "Like a Researcher Stating Broader Impact For the Very First Time" is likely most relevant (https://t.co/HTLSL1h1Vb) 2021-04-07 23:59:07 Finally https://t.co/QwU6ajqknW 2021-04-06 20:40:43 RT @NandoDF: Bravo @kchonyc Highly deserved! https://t.co/A9Uom1aVZf 2021-04-06 19:53:05 RT @josheidelson: Scoop w/ @NicoAGrant & 2021-04-06 19:51:54 RT @L_badikho: The resignation of Samy Bengio is a big loss for Google. Samy co-founded one of the most fundamental research groups in indu… 2021-04-06 17:41:59 RT @RMac18: It took more than a year to report this story. We contacted more than 1,800 US taxpayer-funded organizations listed in data as… 2021-04-06 16:50:23 @ZeerakW @yoavgo lol or/and travel grants for everyone! 2021-04-06 16:38:09 @yoavgo 2021-04-05 21:40:42 RT @DataSciBae: If you read this thread and learned something, check out the full versions of each paper. I rounded them up here: https://t… 2021-04-01 12:58:30 @DavidVidalJD @MarkSendak @m_c_elish Interesting - reminds me of this attempt at Google to make Model cards "easier" : https://t.co/2UyA4sUNiZ 2021-04-01 12:48:44 @DavidVidalJD @MarkSendak @m_c_elish Thank you! 2021-04-01 12:24:16 @DavidVidalJD @MarkSendak @m_c_elish Nice! Though the link in the tweet on regulatory science tools gives an error :/ 2021-03-31 17:43:21 @gofango 2021-03-31 17:42:52 RT @CGAPeterson: Extremely well made Vox video just released featuring many of the big figures in AI Ethics. From Twitter cropping bots to… 2021-03-31 17:42:39 RT @voxdotcom: Ruha Benjamin (@ruha9), author of Race After Technology, explains why AI often fails people of color: "It’s a systemic iss… 2021-03-31 12:48:14 @agstrait @SandraWachter5 lol I'm scared to read this 2021-03-31 12:46:00 RT @black_in_ai: We are excited to announce the launch of the BlackAIR Summer Research Grant Program providing support for AI Research pro… 2021-03-31 12:41:53 @agstrait @SandraWachter5 wow - what's the context for this report?? 2021-03-30 20:27:32 RT @Abebab: Wow! This is huge!!! Well done, @Foxglovelegal Also, reminder that @WFP still works with Palantir, where the fate of vulner… 2021-03-30 14:46:05 RT @shalinikantayya: I'm so incredibly thrilled to announce that @CodedBias will be available to stream globally on @netflix April 5th! It’… 2021-03-30 12:51:27 RT @hypervisible: Such essential journalism, again pointing to the fact that institutions often deploy computational tools against students… 2021-03-30 12:38:28 RT @Aaroth: I wrote a blog post two+ years ago last time folks were discussing bias in data vs bias in algorithms. https://t.co/LQlm9LXCUh… 2021-03-30 07:38:13 @BrianSJ3 Yeah, for sure, I'll keep this in mind for next time! Thanks for bringing this up. 2021-03-30 07:30:10 @nancy_iskander Human bias would be independent of the algo, that's not the case here - the algo makes bias worse, just via interactions rather than model output (https://t.co/6LTeXkadTV). To some, "algorithmic bias" = "algorithmically enabled discrimination", necessitating a systems-level view. 2021-03-30 07:19:43 @BrianSJ3 Yeah, I was wary of using that term when I wrote this - in this situation, I am talking about the same thing, but in a context when a group trusts the algorithm *more* (ie. demonstrates more "automation bias") in situations when making a call in favor of their natural prejudice. 2021-03-30 07:08:09 RT @black_in_ai: Congratulations to @red_abebe , @wsisaac , @shakir_za and @png_marie for contributing to this new book exploring work, de… 2021-03-28 17:20:11 @BetaBlueIS Oh, sorry if this point was unclear. Of course we should intervene! My point was that we shouldn't frame such interventions as "fixes", because it gives people people the excuse they're looking for to abdicate responsibility 2021-03-28 17:01:22 @boazbaraktcs @kareem_carr @JacobBloom31 @KLdivergence @mrtz @AngeleChristin @mawnikr @zephoria @HodaHeidari @mkearnsupenn @Aaroth There's also the fact that regardless of the "bias" definition, certain assertions that feel natural in idealized scenarios (ie "fixes","obvious gender & 2021-03-28 16:57:22 @boazbaraktcs @kareem_carr @JacobBloom31 @KLdivergence @mrtz @AngeleChristin @mawnikr @zephoria @HodaHeidari @mkearnsupenn @Aaroth I agree with a lot of what you say here - from my perspective, Kareem's definition of bias is only appropriate in a certain narrow context (ie data representation bias?) but an expanded definition is required to properly formalize these issues for other situations, and contexts. 2021-03-28 16:42:23 RT @harini824: read @rajiinio's thread! also: the updated version of this paper/fig includes "deployment bias," or harm arising from mistr… 2021-03-28 04:48:01 @AngeleChristin which we discovered is from @harini824's paper here: https://t.co/LTm0qlwhNj 2021-03-28 04:38:34 @AngeleChristin I pulled it from this blog! https://t.co/vLdvffmH8q 2021-03-28 04:37:43 RT @dmshanmugam: one of @harini824’s many beautiful figures! https://t.co/yRBHtlvrbn 2021-03-28 03:24:52 @tdietterich Oh, thank you 2021-03-28 01:15:24 @athundt No worries! Someone else asked and we found it here: https://t.co/LTm0qlwhNj 2021-03-28 00:28:30 @JFPuget Well, not everyone can be correct about everything. These misconceptions aren't at all unique to Kareem - many statisticians make similar assertions when first learning about the algorithmic fairness space, and it even took the community quite a while to get to this point. 2021-03-28 00:14:08 @kareem_carr I hope this didn't feel like too pointed a call-out, that's not my intention (I really like your memes!) but wanted to flag that the framing in the original thread could easily lead to this confusion, even if that wasn't the intent. 2021-03-28 00:12:42 @kareem_carr Yeah, I mention in my thread that we're interpreting that term "algorithmic bias" differently - however, my point at the end is that reducing the conversation of algorithmic discrimination to statistical bias gives people the excuses they're looking for to escape responsibility. 2021-03-28 00:05:55 @timnitGebru lol there's a whole syllabus of content one could develop on "What Twitter keeps forgetting about algorithmic bias" 2021-03-28 00:00:32 @timnitGebru lol I was thinking of you when I wrote this. This is for reference for us to use later, when the conversation inevitably comes up again. 2021-03-27 23:55:38 @ryanbsteed Not sure if this is the original, but I grabbed it from Harini Suresh's blog post on "The Problem with Biased Data" https://t.co/WdPwPK4w0c 2021-03-27 23:46:02 Sadly, people use these misconceptions to escape responsibility (ie."I fixed the data bias, there's nothing I can do.") so it's important to be careful. Algorithmic discrimination requires socio-technical systems level thinking - that should affect how we think about all of this. 2021-03-27 23:46:01 One culprit for these common misconceptions is the use of that term "bias". Some hear "algorithmic bias" and map this to "statistical bias" (as in the simple notion of bias vs. variance). Others hear it and think of algorithmically enabled discrimination and harm. Both are right. 2021-03-27 23:46:00 There's also a reality of various tradeoffs (ie. diversifying data may require privacy violations, measuring fairness could increase liability, etc.) & 2021-03-27 23:45:59 3. Race & Here's a recent @FAccTConference paper about this: https://t.co/yQ0Ju9A8L8 https://t.co/InS5FwP8rs 2021-03-27 23:45:58 2. Much of the de-biasing work makes it clear that algorithmic design choices can lead to *more fair* outcomes (w/ a "fixed" dataset) so it shouldn't be surprising that algorithmic design can also lead to *less fair* outcomes. @sarahookr explains it here: https://t.co/Wc9RUJzouJ 2021-03-27 23:45:57 For example, "automation bias" occurs when just the introduction of an algorithm results in increasing the bias of human discretion (ie. model predictions being perceived differently in one context vs. another, leading to biased outcomes for minorities). https://t.co/kL8FWKbojT https://t.co/fEPLfHImr7 2021-03-26 18:42:24 RT @emilymbender: This looks excellent -- and I'm particularly impressed with this innovation, meant to get people thinking about these que… 2021-03-26 17:20:07 RT @mmitchell_ai: "...authors should be rewarded, rather than punished, for being up front about the limitations and potential negative soc… 2021-03-26 16:22:38 @jackclarkSF @geomblog yup, and better edits of stunt double work! Alarmingly thin line between "this is disinformation" and "this is just part of the art" 2021-03-26 16:08:20 @geomblog As far as I know, main use cases are data augmentation (ie. add generated images to supplement a too-small dataset), marketing copy (ie. generate images with fake models, rather than real ones) and animation (ie. GANs can generate in-between frames to increase resolution, etc.) 2021-03-26 16:04:08 RT @NeurIPSConf: Introducing the NeurIPS 2021 Paper Checklist! https://t.co/SjMkX6JMVR 2021-03-26 13:43:57 RT @UMDscience: TODAY: Join us at 4pm ET for our @CodedBias panel discussion. Panelists: @rajiinio who's featured in the film, @drturnerlee… 2021-03-25 22:34:30 RT @timnitGebru: "A Google spokesperson said that, over the past 15 years, the company has furnished over 6,500 academic and research grant… 2021-03-25 02:16:54 I'm really proud of the author for articulating her experience so beautifully - can't even imagine what it took to be so open. Please keep in mind our Asian colleagues still processing & h/t @SanhEstPasMoi for posting! 2021-03-25 02:16:53 Love this essay by Renee Chang, responding vulnerably to the shooting in Georgia. I've been there before, it hurts so badly. Attacks on our communities are attacks on us as individuals - the fear, anger & https://t.co/kv7lb7khWp https://t.co/Fyc4vTO5XK 2021-03-24 19:04:54 RT @Jane24477: Facebook and Amazon are now corporate America's two biggest lobbying spenders. my new report @Public_Citizen finds that Bi… 2021-03-24 16:14:27 @marylgray @morganklauss @amironesei Thank you, Mary! 2021-03-24 13:48:42 I was so shocked to read this. Please tell me @ozm will continue in some other form off the platform, we need that kind of reporting! (+ I think @hackernoon successfully emigrated off a couple years ago?) https://t.co/FnIKAzKvJH 2021-03-24 08:20:46 @math_rachel @morganklauss @amironesei Thanks for reading, Rachel! 2021-03-24 01:19:50 @ZeerakW @Abebab 2021-03-24 01:12:21 @josephdviviano @sh_reya Don't mean to nag you, it's just that I see that stereotype a lot (ie. High GPA people are uncreative), and think it can become just as ridiculous as those saying a low GPA somehow means incompetence. It's a hack metric, it doesn't mean much of anything. People should ignore it. 2021-03-24 01:08:17 @josephdviviano @sh_reya Sure but some well rounded, disadvantaged, busy people also get high GPAs. The only thing a high GPA means is that a person was taught to care about that metric & 2021-03-24 00:46:43 @ZeerakW Let's start a GoFundMe so we can both attend @Abebab's next talk 2021-03-23 23:23:43 @josephdviviano @sh_reya Hm, I don't know if this is fair. It can take grit & 2021-03-23 23:04:52 RT @dustinvtran: I'm so appreciative that ML is at a state where open-source code, freely available conference videos/proceedings, and now… 2021-03-23 23:02:31 @_KarenHao @Abebab 2021-03-23 23:02:11 I'm very proud to have a famous friend like @Abebab https://t.co/j48qKK19As 2021-03-23 19:13:32 RT @HughLangley: Last week, @luke_stark turned down a $60k Google research scholar award because of the company's treatment of @timnitGebru… 2021-03-23 12:46:39 @schock 2021-03-22 22:51:25 RT @jovialjoy: TONIGHT on @PBS As we celebrate #WomensHistoryMonth, I am so honored to share the screen with these phenomenal women @timn… 2021-03-22 01:35:44 RT @JayAlammar: What are inductive biases? Can models make different predictions when trained on the same data? @RTomMcCoy distills the co… 2021-03-19 18:20:46 RT @mmitchell_ai: @rachelmetz Yeah, it means so much. @timnitGebru and I both made choices in line with AI ethics, and so lost our income.… 2021-03-19 18:18:15 @luke_stark wow - this is unbelievable leadership. Thanks for making what must have been a very difficult decision in order to advocate for this. 2021-03-19 18:17:35 RT @luke_stark: Last week I found out I'd been selected for a Google Research Scholar award. Today I declined it. https://t.co/7LDmGKlwy8 2021-03-19 13:41:41 Someone please help me with this too Gmail is a nightmare to work through at the moment. https://t.co/WBDhpYmScE 2021-03-19 13:40:59 RT @tiffani: Does anybody know how to scale back Gmail’s spam filters? It is sending emails from people I’ve been corresponding with for… 2021-03-19 13:40:38 @KLdivergence @mer__edith @tiffani I literally got mad about this happening to me right now - a link for recording a talk ended up in spam 2021-03-18 16:52:22 RT @Abebab: Clearview AI is literally phrenology rehashed in a digital form https://t.co/JQDTNnrIJA 2021-03-18 12:15:21 RT @schock: Good morning! Please circulate this awesome opportunity! https://t.co/TeZooBkEgc 2021-03-18 09:16:35 @trelsco Hm I don't think their users are their customers. They do provide value to their customers (ie. Advertisers) 2021-03-18 09:03:46 RT @yisongyue: Growing up in Chicagoland in the 80's & I've been called chink and slanty-eye… 2021-03-17 18:40:47 Oh, this is happening tonight! https://t.co/NeHIvOmSME 2021-03-17 17:25:02 @psettel @RLerallut I do agree with your earlier point though, that Innovation and Regulation can have this two-way arrow influence, and mutually beneficial interaction. I also really find that framing of Awareness, Alternatives and Regulation to be helpful, so thanks for introducing that framing! 2021-03-17 17:23:19 @psettel @RLerallut Ahaha I'll peacefully disagree on that one. I've lost faith in meaningful self regulation some time ago, but I do think companies can be pressured into acting well without regulation through strategic litigation & 2021-03-17 17:09:51 @psettel @RLerallut And I think actual regulation will take time, but some level of legal liability should apply to the issues we care about, and at minimum advocates should be organizing with the understanding that most companies will need to have constraints enforced on them in order to improve. 2021-03-17 17:08:25 @psettel @RLerallut I agree awareness is essential first but I don't think alternatives need to come before regulation. There's already examples in privacy/security where policy restrictions (based on public needs, to address problems w/o an available solution) actually spurred innovation. 2021-03-17 14:57:34 @LeonYin 2021-03-17 14:57:20 RT @LeonYin: Today I learned national hate crime stats are undercounted and unreliable. https://t.co/e44gSbo9fJ 2021-03-17 12:57:43 RT @DrIbram: Locking arms with Asian Americans facing this lethal wave of anti-Asian terror. Their struggle is my struggle. Our struggle is… 2021-03-17 12:56:00 I found this to be devastatingly accurate https://t.co/IHLa3DghDS 2021-03-17 12:54:55 This is really good point. And when the understanding of the technology is limited, policymakers will often rely on corporations to break down the functional details for them, making them even more vulnerable to the corporate PR narrative. https://t.co/VToxkJAkYa 2021-03-17 11:55:50 "Only the unloved hate I remind myself of that whenever things like this happen. Sending love & 2021-03-17 04:08:44 @EricMeyersonSF And those consequences could be as simple as "lies don't get algorithmically amplified". Right now, there's nothing - there's no regulation even requiring any scrutiny and def nothing penalizing those that create & 2021-03-17 04:04:18 @EricMeyersonSF Yes, this is a really good point. Ironically, Ailes lobbied against legislation at the time (ironically called "The Fairness Doctrine") in order to escalate Fox to it's current state. I'm partial to the take that MSM or not, there should be consequences for putting out misinfo. 2021-03-17 03:58:02 @go_kerem_go For sure! 2021-03-17 03:25:00 RT @_KarenHao: I write words for a living and yet cannot find the right ones to describe the mix of grief, fear, and loss of seeing the new… 2021-03-17 03:24:55 @_KarenHao My heart breaks for you - hang in there. 2021-03-17 03:01:35 @go_kerem_go I think there's a lot of reasons fairness is a palatable problem to work on for industry researchers but it doesn't make sense for AI policy practitioners to obsess about bias. I think policymakers can certainly do better to think about regulations needed for other harms as well. 2021-03-17 02:41:18 @geomblog I understand & 2021-03-17 02:37:06 @Miles_Brundage This is a fact. Never felt more strongly the need to expand our vocabulary on AI harms in pretty much every context they're discussed. 2021-03-17 02:16:04 And that leads me to the thing I don't understand: that panic Zuckerberg feels for "conservative bias" and upcoming algorithmic discrimination regulation - why doesn't he feel that for misinfo? If we care about that problem, why don't we have regulation to address those concerns? 2021-03-17 02:16:03 This happens with other companies too. Uber has its large Fairness Working group, but funds campaigns to strip drivers of their worker rights. Amazon funds an NSF fairness grant but refuses climate friendly innovation for data centers & 2021-03-16 23:49:54 RT @PBS: MIT Media Lab researcher @jovialjoy makes a startling discovery: Machine learning algorithms are only as unbiased as the humans an… 2021-03-16 23:06:51 RT @timnitGebru: I wrote exactly the same thing. The way they tried to discredit me was exactly how they tried to discredit you and Joy & 2021-03-16 22:52:32 @camillefrancois @FastCompany @Graphika_NYC @schock @jovialjoy Grateful to know you rockstars 2021-03-16 21:38:59 @camillefrancois @FastCompany @Graphika_NYC Amazing! Congrats 2021-03-16 21:29:44 Thanks to @CadeMetz & 2021-03-16 21:25:22 I finally read this NYT article, and was shocked to discover I'm heavily featured. But it makes sense? What's happening now with Google isn't divorced from what happened with Amazon, and whatever will happen to whoever needs to be challenged in the future. Same story, ongoing. https://t.co/66x0nasosu 2021-03-16 20:57:00 @nsaphra @chrisalbon I appreciate the alliteration there 2021-03-16 20:06:38 @geomblog @ruchowdh @Combsthepoet oh, interesting! Well Beethoven was certainly much more prolific aha 2021-03-16 19:58:05 @geomblog @ruchowdh @Combsthepoet Why do you call it Beethoven vs Mozart? 2021-03-16 19:49:28 @geomblog @chrisalbon hahaha this is the only logical response... 2021-03-16 19:23:37 RT @alexhanna: I am glad that @Abebab and @vinayprabhu have pointed out this pattern of citation erasure from ImageNet authors. There's a w… 2021-03-16 19:11:46 @chrisalbon In a clumsy attempt at anti-hype, I once referred to an ML model as a "data-defined model" 2021-03-16 18:26:35 How anyone could think that they could mess with @Abebab and get away with it is beyond me... https://t.co/DfOHHKBT2m 2021-03-16 17:22:34 RT @mozillafestival: Live tomorrow from #MozFest: a panel discussion of acclaimed documentary #CodedBias With @rajiinio, @CummingsRenee,… 2021-03-16 17:16:31 RT @tsimonite: Google is running an invite only robotics workshop this week. Two academics withdrew in protest of the company's treatment o… 2021-03-15 19:26:57 @mmitchell_ai @DiverseInAI @timnitGebru @AJLUnited @jovialjoy @ZoubinGhahrama1 Nonsense! More like "#1" to me 2021-03-15 18:53:34 So far, we've been learning so much about how practices from the security space can inform our approaches to identifying, prioritizing and documenting algorithmic harms. Beyond grateful to be working with @schock, @camillefrancois, @jovialjoy & 2021-03-15 18:48:42 Me & "If we had a harms discovery process that, like in security, was ...robust, structured & https://t.co/ADbdDgca0v 2021-03-15 18:37:30 RT @daphneleprince: I spoke to @rajiinio about the fascinating work she is doing with @AJLUnited to apply bug bounty models to the detectio… 2021-03-15 18:36:51 RT @mathbabedotorg: Amazing article today with Timnit, Meg, and Deb prominently featured! https://t.co/V2J9WzOYnx @rajiinio 2021-03-15 18:36:37 RT @mozilla: #MozillaFellow @rajiinio is researching ways to apply the models that underpin bug bounty programs to detect #AIBias. More o… 2021-03-15 18:36:22 RT @schock: Excited to see the hard work of @rajiinio featured in this @ZDNet article! Shout outs to the @AJLUnited CRASH project (Communit… 2021-03-15 12:50:23 @jachiam0 Spider man? 2021-03-15 12:38:27 RT @JordanBHarrod: First issue of my newsletter is live on Substack! Featuring: a quick intro to me and some life updates, interesting v… 2021-03-15 12:38:02 @JordanBHarrod Love this! I'm about to start one of these myself (eventually...), and yours is the prettiest/best formatted I've seen so far 2021-03-12 16:25:09 @ghoshd Aha, no worries at all - here is info on signal (https://t.co/x1J4Ie43WF). It's an alternative messenger app company, that really values privacy! 2021-03-12 16:21:51 @ghoshd Also, before anyone says anything - yes, I'm aware Whatsapp is part of Facebook, but they've had to fight hard internally to maintain encryption and a commitment to their values on privacy. Facebook has been ready to compromise on this from the beginning. 2021-03-12 16:19:53 @ghoshd Mozilla? Whatsapp? Signal? Subscription-based platforms, that opt for a different business model? I'd even say Twitter, on occasion. My point is that there are some companies that don't make the extent of the compromise that Facebook has made to keep up disinfo & 2021-03-12 16:16:03 I tend to disagree with this. Yes, the business model is flawed. Optimizing for advertising clicks & 2021-03-12 13:06:16 @widdr @_KarenHao @schrep @CaseyNewton They were only directed towards & 2021-03-12 13:01:43 @widdr @_KarenHao @schrep @CaseyNewton Fair. Facebook, as a company, like Google as a company, doesn't care about anything at all - it's a capitalist institution optimized for profit. That being said, there are individuals at Facebook that do care & 2021-03-12 01:36:19 @_KarenHao @schrep @CaseyNewton The article does a great job clarifying that although progress on these issues was meaningful, it missed the point in addressing the primary way Facebook is understood to cause harm (ie. misinfo). Harm reduction is not just about minimizing liability, and they misunderstood this. 2021-03-12 01:32:54 @_KarenHao @schrep @CaseyNewton I think it's a challenging framing for them to process because Facebook does indeed have dire fairness/bias issues (ie. they were sued by HUD/ACLU for biased ad delivery for housing, jobs). Facebook focused on those issues, as situations most directly tied to ongoing legal threat 2021-03-12 01:08:27 RT @Aaron_Horowitz: I've been feeling down about the potential of algorithmic audits as of late, but then this happened! It was more of a t… 2021-03-11 21:04:54 @mmitchell_ai @rachelmetz @timnitGebru I want to reassure you it was important work, and that will only become more clear over time. There's no way you could have known what was coming. Rooting for you all! 2021-03-11 21:01:36 RT @frecklesforgood: Fascinating discussion between @rajiinio and @camillefrancois asking what we can learn from #CyberSecurity to identify… 2021-03-11 20:23:06 RT @mozillafestival: Streaming in five minutes on #MozFest live: Hunting Biased Algorithms: A #DialoguesandDebates conversation with @r… 2021-03-11 19:28:18 RT @rachelmetz: new from me: an in-depth look at months of chaos in google's ethical ai group, and how it has reverberated throughout the A… 2021-03-11 17:17:02 RT @djleufer: Excellent thread from @AINowInstitute on the fantastic work on AI that's been done from outside the Euro-American context Fe… 2021-03-11 13:55:36 RT @JuliaAngwin: Facebook is a mass personalization tool: everyone’s feed is different. Today we launch “Split Screen” — a tool that lets… 2021-03-11 00:06:40 RT @FAccTConference: Thanks organizers, volunteers, attendees, authors, participants & 2021-03-11 00:00:14 @Abebab every time Don't think we've ever actually successfully met for 15 mins! 2021-03-10 23:58:43 RT @carolinesinders: @rajiinio I dig citizen browser but also the work or @_vecna and @WhoTargetsMe is pretty grand in this area too! 2021-03-10 23:58:19 @carolinesinders @_vecna @WhoTargetsMe Oh nice - I wasn't aware, thanks for sharing! 2021-03-10 21:55:26 T'was much fun to be publicity co-chair with @Abebab for @FAccTConference this year! Thanks @m_c_elish, @wsisaac & Also, yes, sorry for the gifs, but this is now part of the official brand guide. 2021-03-10 20:34:23 RT @timnitGebru: It is surreal to watch the talk for our paper by @mcmillan_majora and @emilymbender at @FAccTConference. Never imagined wh… 2021-03-10 20:03:36 RT @FAccTConference: In Stream 2: 1) Censorship of Online Encyclopedias: Implications for NLP Models by Eddie Yang & 2021-03-10 19:40:13 RT @FAccTConference: "What does the future of this research community look like both epistemically & 2021-03-10 19:39:39 RT @AINowInstitute: Great article by on the vital (& 2021-03-10 19:11:35 RT @FAccTConference: STARTING SOON IS THE #facct21 TOWN HALL. (Live on Stream 1) This event will provide attendees with the opportunity… 2021-03-10 19:02:41 @AutoArtMachine @JuliaAngwin lol replace "police" with "tech company" and that's pretty much our situation here 2021-03-10 19:00:57 RT @__lucab: “All data is terrible” @JuliaAngwin Why is this so true @FAccTConference #FAccTConference 2021-03-10 18:54:44 @AutoArtMachine @JuliaAngwin Yes, and the example she gives is very telling of this - when WSJ had to collect the database on the racial breakdown of police killings since law enforcement was not incentivized to create such a dataset. 2021-03-10 18:50:45 Love how she doesn't sugarcoat any of the work involved in this. "We don't spend a lot of time 'asking for access' - corporate provided data is just like leaked data, it's politically motivated. We will put in requests but can be fairly adverserial in our data collection approach 2021-03-10 18:14:00 "Data can help sway a debate" - mentions how their work is public service, to contribute "new facts & 2021-03-10 18:12:54 Wow @JuliaAngwin at #facct21 mentions "treating engineers as journalists, with a different skillset", mentions the goal of wanting to get to a 1:1 ratio at The Markup, where multiple engineers can work with multiple reporters to collaborate on great stories & 2021-03-10 18:04:06 RT @FAccTConference: Next, our final keynote speaker is Julia Angwin @JuliaAngwin (The Markup) speaking about "Algorithms, Accountability,… 2021-03-10 18:02:06 RT @yuvalmarton: Great example of how to do #DataScience and #AI right There’s even an #NLProc candy here... New POS tagger for certain T… 2021-03-10 17:54:13 RT @SeeTedTalk: #FAcct2021 panel moderator @vinodkpg making the important observation that teaching #AIEthics via a series of invited speak… 2021-03-10 16:09:20 @WellsLucasSanto oh wow, thank you so much for live tweeting I'm always energized by your natural enthusiasm for this work! 2021-03-10 16:08:06 By the way, "Artificial Intelligence and Inclusion: Formerly Gang-Involved Youth as Domain Experts for Analyzing Unstructured Twitter Data" is a great paper from this lab that ML people need to read. Exemplifies participatory ML & 2021-03-10 15:53:58 I was referring to this: "What Do We Teach When We Teach Tech Ethics?: A Syllabi Analysis" https://t.co/gNNLQD1n5k + There's this older paper by Deborah Johnson - "Who should teach computer ethics and computers & that gets right to the point. https://t.co/5il1uPh7pu 2021-03-10 14:59:37 RT @WellsLucasSanto: @rajiinio: "The expansion of the AI field is required to solve the AI ethics problem, but this exclusionary behavior n… 2021-03-10 14:58:19 RT @WellsLucasSanto: Through their survey of courses, they found that participants of the CS discipline tried to isolate themselves other d… 2021-03-10 14:56:22 This work is lovely. "Prior work has looked into the perception of algorithmic decision-making from the *user's* point of view. In this work, we investigate how students in fields adjacent to algorithm development perceive algorithmic decision-making." https://t.co/nHbS2go1Lt https://t.co/DxEpW7Ge08 2021-03-10 14:54:49 RT @pinbarlas: "I agree with the decision but they didn't deserve this" - A paper in #facct2021 on how future developers (current students)… 2021-03-10 14:52:57 RT @pinbarlas: We're presenting this paper at #facct21 #facct2021 in about 45 mins on Track Two! Join us! https://t.co/ysOKxmTVsa 2021-03-10 14:27:58 RT @WellsLucasSanto: .@rajiinio starts the talk off by mentioning that there's an AI ethics crisis happening & 2021-03-10 13:22:17 RT @mdekstrand: Really happy to see attention to measurement at FAccT. Let's do a lot more of this please . I personally think that some o… 2021-03-10 12:28:13 RT @Andrew__Brown__: We recently held our “Thinking Through and Writing about Research Ethics Beyond “Broader Impact” ‘ tutorial at @FAccT… 2021-03-10 12:24:42 RT @j2bryson: @rajiinio @bgeurkink Exactly. Similarly, trust is the wrong metaphor for AI https://t.co/N0wiePRMR0 see also the transpare… 2021-03-10 12:23:54 RT @LauraC_rter: "Educate students on frameworks of interventions based on existing problems, not anchored to the existing skills of those… 2021-03-10 03:21:09 @kareem_carr how do you differentiate between inference and prediction? 2021-03-10 03:20:00 RT @AINowInstitute: A panel deep dive into #landlordtech by @erin_mc_elroy and their work w/@antievictionmap: "Currently, there are over 2,… 2021-03-10 03:19:07 RT @d_malinsky: @rajiinio @ziebrah @david_madras Hi @rajinio pardon the shameless self-promotion but I agree with you! (at least abt the se… 2021-03-10 02:27:03 @d_malinsky @ziebrah @david_madras @rajinio Nice! Thanks for sharing 2021-03-10 01:33:00 RT @berkustun: @rajiinio @geomblog @Aaron_Horowitz @ziebrah @david_madras Every prediction can be explained. This includes: - Predictions… 2021-03-10 01:27:44 @aselbst @ziebrah @david_madras @s010n @manish_raghavan Though I'm still kind of thinking that explainability people don't yet focus on causal inference the way fairness people do and that part doesn't make sense to me lol This is kind of a point your own work makes though (thinking of your last year's Facct paper on explainability) 2021-03-10 01:26:26 @aselbst @ziebrah @david_madras @s010n @manish_raghavan For sure - it makes sense for them not to talk. Explainability and fairness are framed as very different goals. What's interesting is that both communities converged on the need to re-define their problem wrt causal inference. It implies some less obvious connection between them. 2021-03-10 01:20:05 @andrewthesmart @ziebrah @aselbst @david_madras lol moral of the story is that we all need to read your paper 2021-03-10 01:18:05 @aselbst @ziebrah @david_madras @s010n @manish_raghavan Yeah, this is kind of wild. It's possible there's some wheel re-invention happening here? There's clearly some cultural divide (ie. non-overlapping communities) responsible for that + now I'm scratching my head about which of how this confusion/divide plays out in XAI policy 2021-03-10 01:11:31 @geomblog @Aaron_Horowitz @ziebrah @david_madras @berkustun its the good fight 2021-03-10 01:10:07 @ziebrah @aselbst @david_madras ahahhaha oh lord 2021-03-10 01:09:46 @aselbst @ziebrah @david_madras personally more familiar with the latter and not so much with the former though I guess I can imagine. Either way, what I meant earlier is that although explainability clearly has some causal expectation, the causal inference crowd I'm exposed to often frames goals wrt fairness 2021-03-10 01:07:26 @aselbst @ziebrah @david_madras oh interesting - what's the interplay between "counterfactual explanation" and "counterfactual fairness"? lol 2021-03-10 01:01:20 @mdekstrand @aselbst @ghadfield @BrownSarahM @ziebrah @david_madras @s010n Unless you're confirming that none of the possible explanations are awful? The idea of multiple acceptable causes makes me kind of uncomfortable, almost like something is underspecified and now there's room for a cop-out. 2021-03-10 01:00:07 @mdekstrand @aselbst @ghadfield @BrownSarahM @ziebrah @david_madras @s010n That's the thing for me - causal explanations and "causal inference for fair outcomes" are super murky, and look almost like the same problem when you think about it. Also, the idea of multiple explanations/possible causes is interesting, but I worry it's not necessarily useful 2021-03-10 00:57:23 @mdekstrand @aselbst @ghadfield @BrownSarahM @ziebrah @david_madras @s010n Thanks for sharing - I'll check it out! 2021-03-10 00:56:37 Someone please fight Aaron on this. (I think he's right tho) https://t.co/rhdC1GoJXn 2021-03-10 00:55:51 @Aaron_Horowitz @ziebrah @david_madras I love how meaningful Facct participation necessitates starting beef over the policy interpretations of technical terms lool 2021-03-10 00:54:53 @Aaron_Horowitz @ziebrah @david_madras 2021-03-10 00:53:45 @aselbst @mdekstrand @ghadfield @BrownSarahM @ziebrah @david_madras @s010n Next year's tutorial lol 2021-03-09 18:09:12 @salome_viljoen_ @RDBinns @lilianedwards @mikarv You should email the reporter? That's a frustrating oversight. Also, they got it so wrong aha 2021-03-09 18:05:47 @mmitchell_ai @KLdivergence samesamesame 2021-03-09 18:05:20 RT @FAccTConference: STARTING NOW Our keynote speaker for the day will be Katrina Ligett (Hewbrew University) speaking "In Praise of (Flaw… 2021-03-09 14:36:08 RT @AJLUnited: We are honored to be among so many incredible companies that made @FastCompany's annual list of the World’s Most Innovative… 2021-03-09 14:35:50 RT @TessaDarbyshire: Thank you for an excellent Q& 2021-03-09 14:35:10 RT @_pmkr: The #facct2021 talk I recorded on our work, "An Action-Oriented AI Policy Toolkit for Technology Audits by Community Advocates… 2021-03-09 11:59:38 RT @FAccTConference: Paper sessions starting now! We have 2 Streams in parallel, each with 4 papers. In Steam One: 1) Narratives and Count… 2021-03-09 01:38:57 Much thanks @AngeleChristin for facilitating such a great discussion - I learnt a lot! 2021-03-09 01:37:28 Also liked @autopoietic's closing words: “The creation of knowledge and the generalizability of it matters. As lived experiences, value systems and knowledge systems from diverse sources come together, hopefully we learn from each other and we see that in the next few years." 2021-03-09 01:36:36 “Find algorithms that benefit people on their own terms.” - @undersequoias 2021-03-09 01:25:49 RT @vmsoutherland: Some work to share. My soon-to-be-published article “The Intersection of Race and Algorithmic Tools in the Criminal Lega… 2021-03-09 01:13:22 Wow - this is an amazing discussion. So many practical examples being shared piecing together what accountability can mean in a variety of contexts! https://t.co/huBMwWp8XY 2021-03-09 00:43:03 RT @BrownSarahM: Our FAQ in the #facct21 conference guide is being built as we go. As attendee questions come in, we're taking care to arch… 2021-03-08 22:07:39 @KLdivergence Agreed - not testing for this is completely inexcusable at this point. I think I said "oops" only because I'm kind of embarrassed for those involved lol 2021-03-08 22:00:58 Ops - not again! https://t.co/qfPCmEV9AT 2021-03-08 20:53:24 RT @timnitGebru: My sisters. https://t.co/g4AS5uOY3D 2021-03-08 20:47:34 @sedyst @alixtrot @MarzyehGhassemi Yes, ditto - loved your facilitation! 2021-03-08 19:55:10 @timnitGebru 2021-03-08 19:53:27 Love these women https://t.co/5NIJkgtneH 2021-03-08 19:52:59 RT @AJLUnited: Happy #InternationalWomensDay! In celebration, meet the Black women who continue to inspire & 2021-03-08 18:36:41 RT @baxterkb: "We like to have conversations around accountability & 2021-03-08 18:21:06 RT @wsisaac: So excited to have @alixtrot moderating the panel today!! She is IMO the absolute *best* at facilitating and guiding complex… 2021-03-08 18:19:18 RT @KLdivergence: On deck to speak at #FAccT2021 is @YESHICAN. I've heard her speak before, and she is fantastic. Tune in now! 2021-03-08 18:03:30 RT @baxterkb: "Trying to replace public health with a machine learning model, it doesn't make any sense." - @rajiinio 2021-03-08 18:02:57 RT @baxterkb: "It becomes incredibly difficult to remove yourself from these systems because you need it to be healthy or to live. There a… 2021-03-08 18:01:13 RT @TessaDarbyshire: "In the FAccT space, we like to talk about fairness and accountability... When it comes to the impact on real lives, w… 2021-03-08 18:00:37 RT @gleemie: How do we get regulation to be independent of companies so there can be actual accountability? Companies want to have transpar… 2021-03-08 16:52:23 RT @kenandoesmath: "Data isn't just numbers. It's people and their stories." Social justice will come from the combination of quantitative… 2021-03-08 16:08:12 @candacerossio Yeah. And in addition to lives lost, there's businesses lost, community and relationships lost. It's a lot. I'm increasingly worried we'll never really be given enough time to process, even afterwards. 2021-03-08 16:01:39 RT @m_c_elish: Today's #FAccT2021 plenary sessions focus on health, data, and in/equity Join @YESHICAN on @Data4BlackLives in discuss… 2021-03-08 15:38:42 It's crazy to me how little anything has stopped during the pandemic and how little time has been dedicated to mourning and reflection. I wonder at what point we're actually going to acknowledge how much we've lost, and how inhumane we've been, just in order to "keep going". 2021-03-08 15:32:44 I wrote "The Discomfort of Death Counts" (https://t.co/uxbfJlTWBU) when the US COVID toll had just hit 100,000 & Now, we're past 500,000 dead. I almost didn't notice the headline. 2021-03-08 12:15:15 RT @FAccTConference: Paper sessions begin today!!! We also have an amazing lineup: a keynote on "Health, Technology and Race" from @YE… 2021-03-08 00:59:45 RT @le_roux_nicolas: @timnitGebru Replies I got when raising google issues: - "You see everything with a negative lens." - "Have you tried… 2021-03-07 23:05:37 By the way @kharijohnson this is the paper where we cite you (in our intro)! 2021-03-07 22:59:39 Wow. Just spotted this. I spend way too much time on Wikipedia so this is truly the highest honor. Thank you! https://t.co/vD902Mhvb7 2021-03-07 22:58:08 RT @catherinehyeo: 2/? Working alongside @hessiejones @volhalitvinets for the @Women_AI_Ethics project, it was an honor to create the page… 2021-03-06 19:36:20 RT @IsmaelKherGar: Such a good read! This paragraph’s references look like GOLD. (h/t @yoyehudi) You Can't Sit With Us: Exclusionary Pedag… 2021-03-06 00:08:35 @SerenaOduro @VinhcentLe @alicexiang Thank you! 2021-03-05 23:26:00 @SerenaOduro @VinhcentLe I couldn't get into any sessions Is there a recording of this / link to your work? 2021-03-05 23:20:40 @hannawallach @krob @STurkle @joannagoode13 @morganklauss @amironesei lol yeah, we cite it in this paper and talked through that work a lot while drafting. 2021-03-05 22:45:31 RT @hima_lakkaraju: Here is the link to our @FAccTConference tutorial on "Explainable ML in the Wild: When Not to Trust Your Explanations"… 2021-03-05 22:43:51 RT @quarantiain: Yes! Transparency without scrutiny and accountability is us being given a front row seat to our own funeral https://t.co/7… 2021-03-05 22:41:59 RT @tsimonite: Sprinkling a few ethics classes into CS courses isn't enough to keep AI on the rails, say @rajiinio @morganklauss @amironese… 2021-03-05 20:18:00 @DataSciBae Yes, I totally agree. And I'm so glad you said something - this honestly doesn't get pointed out often enough! 2021-03-05 18:59:11 RT @m_c_elish: I am simply blown away by all the incredible (50+) #FAccT2021 volunteers. You all are phenomenal! Thank you for making thi… 2021-03-05 18:44:10 @ambaonadventure Yes! lol, you already know it: Disclosure & 2021-03-05 18:34:01 This is one of many reasons I like @hima_lakkaraju's work (I suggest those registered check out her @FAccTConference tutorial "Explainable ML in the Wild: When Not to Trust Your Explanations"). Explanations can't be trusted at face value. Actual accountability requires much more 2021-03-05 18:25:52 These days I think of transparency & The value of transparency is in it's ability to enable system scrutiny & 2021-03-05 18:08:22 @mahimapushkarna @FAccTConference I wish I was there too - unfortunately, the event was already at capacity. Great work though, I really love this project! Thank you so much for working on this 2021-03-05 18:04:11 RT @FAccTConference: NEXT we have another incredible CRAFT session: "Narratives and Counternarratives on Data Practices in the Global Sou… 2021-03-05 17:51:51 RT @verge: Timnit Gebru was fired from Google — then the harassers arrived https://t.co/lGX04xnRLT https://t.co/pwUWvApvUQ 2021-03-05 17:33:58 RT @_KarenHao: You know of labor strikes. But what about data strikes? @nickmvincent, @hanlinliii, @nic_tilly, @snchancellor & 2021-03-05 12:03:56 @histoftech Thank you! 2021-03-05 12:02:55 There's this part of the paper I really like, where @amironesei presents an interesting theory of the value hierarchies that naturally fall out of disciplinary characterizations and categories - inherently giving "hard" fields like CS more power than "soft" HSS disciplines. https://t.co/8nm8yFlaof 2021-03-05 11:53:32 RT @wsisaac: Very excited for CRAFT this year! Remember to check the hub for links to the respective sessions. If you are not attending CRA… 2021-03-05 11:27:06 RT @JulianPosada0: How do traditional disciplinary divisions encourage exclusionary pedagogy in AI Ethics? Disciplines “don't value each ot… 2021-03-05 11:24:56 RT @rachelcoldicutt: This is https://t.co/I8Q9c7JHfF https://t.co/S6uleW5Iua 2021-03-05 04:34:37 @JulianPosada0 @morganklauss @amironesei @cgpgrey Aha neither did we, honestly, but "Humans Need Not Apply" was referenced so many times we had to mention it lol 2021-03-04 23:55:27 Context on why we went out of our comfort zone, as people coming from VERY different academic backgrounds, to collaborate and write this: https://t.co/DuiQflPI30 2021-03-04 23:42:02 RT @WellsLucasSanto: I glanced through the slides for the paper talk (deep diving on all the paper talk videos this weekend), and this is d… 2021-03-04 23:41:06 RT @sherrying: Yes! I was just thinking about some of these ideas on the problem with the dominance of computational approaches in AI and t… 2021-03-04 23:21:30 @WellsLucasSanto Ah, thank you, thank you - so glad you're finding this work interesting & 2021-03-04 23:05:18 Finally, it's out! It's here! In this paper, me, @morganklauss & Check it out: https://t.co/f2ByPGx5q1 https://t.co/WCD9fecmUO 2021-03-04 22:37:27 RT @BrownSarahM: As always, I *loved* @red_abebe 's thinking in her tutorial on human centered mechanism design. That was quite the readin… 2021-03-04 21:20:00 These tutorials were fantastic! Those that didn't have the opportunity during the day should seriously take a moment to check these out. Quite a few were like opinionated surveys of past work. It brought the insight of interpretation to the discussion - I enjoyed that so much! https://t.co/bCT6ZQr6JF 2021-03-04 20:57:33 RT @geomblog: What @BrownSarahM is not saying (and should) is that she's been responsible for much of the very nice social environment that… 2021-03-04 15:13:50 RT @GretchenMarina: @shalmali_joshi_ on how explanations can be used to manipulate or miscalibrate trust, or to change fairness perceptions… 2021-03-04 15:11:06 @mdekstrand @emilymbender @FAccTConference We wanted it to be #facct21 but people started using #facct2021 organically lol 2021-03-04 14:13:13 @KLdivergence Thanks for your service So glad to have met you last year! 2021-03-04 14:11:47 @Abebab also lol is this what your photoshoot was for? 2021-03-04 14:00:49 RT @FAccTConference: Our next set of tutorials are going live now! We have 3 parallel sessions. In Stream One -> 2021-03-04 13:29:39 RT @zacharylipton: The 2021 @FAccTConference has begun and a full day of tutorials with amazing speakers is under way! Register and join us… 2021-03-03 23:07:17 RT @benzevgreen: Just had an incredible time kicking off this year's @FAccTConference with the Doctoral Consortium! Thank you to all of the… 2021-03-03 23:03:01 RT @_KarenHao: I love when AI talks are really just thinly veiled history & 2021-03-03 17:49:21 RT @FAccTConference: ACM FAccT has started with the Doctoral Colloquium today! We have Tutorials lined up tomorrow (March, 4) and CRAFT s… 2021-03-03 14:49:10 RT @emilymbender: More excellent reporting by @kharijohnson on the continuing fallout of Google firing @timnitGebru and @mmitchell_ai http… 2021-03-02 21:46:30 RT @timnitGebru: https://t.co/uKj4ZHzNXa from @financialtimes "The US Congress has been considering an Algorithmic Accountability Act, whi… 2021-03-02 15:50:19 RT @ruchowdh: IDK what event organizer needs to hear this, but if you are asking someone to give a talk, can you PLEASE send out calendar i… 2021-03-02 15:11:04 RT @JuliaAngwin: Most risk scores in the criminal justice system do not use race as an input variable. But major universities are using r… 2021-03-02 01:05:06 RT @BrownSarahM: If you're also excited about @FAccTConference, remember the conference site is already open if you're registered! You can… 2021-03-01 12:51:23 RT @fborgesius: Coooool! Keynote by @JuliaAngwin at @FAccTConference! https://t.co/Q5mdP3k36T 2021-03-01 01:22:19 RT @mmitchell_ai: I abruptly lost my job 9 days ago. Tomorrow I lose my healthcare. This is the "view from the top" for a successful female… 2021-02-28 00:46:18 RT @joonlee: With all of the Anti-Asian sentiment around the world over the last week, things feel like they’re starting to reach a boiling… 2021-02-27 20:46:37 I just want to say that I know a Black employee at Amazon and this is 100% true. We need to remember that in situations like this, real people get very hurt. They deserved better. https://t.co/rVh30Rk6aD via @voxdotcom 2021-02-27 20:18:53 @undersequoias @anotherday____ I've been finding a lot of value in the literature on recalls and how that connects to engineering responsibility. If a product fails to fit certain standards for functionality and due diligence, then it's not fit for the market. Unsure how well this applies to AI but I hope so. 2021-02-27 18:08:15 @ndiakopoulos Yeah, I agree that eng responsibility is part of broader ethical conduct. Strategically, though, I wonder if they should be approached separately - IMO ethics requires a different type of discussion & 2021-02-27 17:41:33 @ndiakopoulos @databoydg @undersequoias @bendotolsen I'm really open to changing my mind here though. I've been introduced to these concepts through engineering design / engineering responsibility, so have a pretty biased understanding. 2021-02-27 17:39:31 @ndiakopoulos @databoydg @undersequoias @bendotolsen Oh thank you for sharing! + hm I guess I was speaking more to the fact that the focus tends to be more on what the engineer has to do and the qualities of the artifact or its design & 2021-02-27 16:55:07 @ndiakopoulos I see engineering responsibility as motivation for framing eng design problems with specific values in mind & https://t.co/8znoZvT8SU 2021-02-27 16:52:34 RT @undersequoias: @rajiinio Anyway there’s also this https://t.co/Rpx73wxw2m 2021-02-27 16:47:51 @undersequoias oohh, thanks for sharing this! 2021-02-27 16:47:16 @undersequoias Yep - we both see the current situation the same way. I guess I think I disagree strategically - I think things will be even more of a mess if we make the tent bigger. I want AI/ML people to do "ethics" properly and to so "responsibility" properly. Right now, they do both poorly. 2021-02-27 16:42:44 @bendotolsen @databoydg @undersequoias @mira_lane @yolandagil Me & 2021-02-27 16:42:09 @bendotolsen @databoydg @undersequoias @mira_lane @yolandagil Aha, I love that! Yeah, I think AI is a strange space, where the need for responsibility is greater than even traditional engineering fields (because - data) but for some reason people are more careless than in any other field... 2021-02-27 16:37:10 @undersequoias Looking at Google, the notion of "ethics" in a corporation is a way more contentious thing. Also, practically, especially in AI, where "engineering best practices" is nebulously defined, there's no notion of "engineering responsibility" that resembles the original concept at all. 2021-02-27 16:34:05 @undersequoias Yep - and Google did have both a "Responsible Innovation" & 2021-02-27 16:24:47 @undersequoias There's certainly overlap in concerns, but accountability is framed very differently. Also, I will admit the AI/ML space is very weird, and people use buzzwords in all kinds of ways. It's not uncommon for someone to use these terms incorrectly, especially when they don't apply. 2021-02-27 16:23:01 @undersequoias Ethics takes on more controversial & 2021-02-27 16:20:58 @undersequoias Respectfully, I disagree. Engineering responsibility traditionally means creating a product that's functional & 2021-02-27 16:08:39 @bendotolsen @databoydg @undersequoias @mira_lane Yep - I also think the AI/ML space is wierd. The way these concepts are interpreted and handled in this space is inconsistent in certain important ways from the original ideas. 2021-02-26 18:16:43 For reference, this is the children's book the title of our paper is based on - unbelievable parallels between how the wrong-headed attempt to fit "everything in the whole wide world" into a museum and the silliness of when we try to do the same with our benchmarks. https://t.co/dqSacgWCw4 2021-02-26 18:16:42 This was an incredibly low-key release (still very much work in progress, please provide feedback!) but it led to me making the slides I am still the most proud of. https://t.co/r5b4VeL8er https://t.co/ZT9lWzFAqT 2021-02-26 16:25:58 RT @kschwabable: I've been working on this story about Google pushing out @timnitGebru for the last 3 months. I spoke to 25 people involved… 2021-02-26 14:32:00 RT @FastCompany: In the field of AI, a handful of giant companies are able to direct the conversation, determine which ideas get funding, a… 2021-02-26 14:14:46 RT @kchonyc: https://t.co/eslcM1IO0v 2021-02-26 13:01:13 @mutalenkonde Actually speechless this time lol if anything they should be surveilling all the white collar criminals walking around Wall Street. 2021-02-26 12:58:29 OF COURSE THEY LAUNCH THIS IN THE BRONX ‼ https://t.co/qhhDGXS2Eo 2021-02-26 12:33:25 RT @FAccTConference: FAccT begins next week! If you haven't registered already, make sure you do so https://t.co/BOoUJgdDdr We are looking… 2021-02-26 08:44:03 RT @william_fitz: why hasn't a tech reporter written an article about timnit's advocacy for herself? it's incredible. she's fierce & 2021-02-26 03:49:27 @Combsthepoet @BigDataMargins @WesternU lol there was a grimace on my face the whole time you were sharing those horrific stories. 2021-02-25 23:47:56 RT @JesseDodge: This is a big reproduction of results from a wide variety of papers. Their conclusion is that most modifications to transfo… 2021-02-25 17:18:43 Still can't believe what happened. It's unreal. @mmitchell_ai fought for me. She would remind me to speak up & @timnitGebru hypes me up every time. I wanted to quit, she's the reason I didn't. https://t.co/eJ3Y8Wn2cH 2021-02-25 13:34:00 @math_rachel @jeremyphoward Good luck on your next adventure, Rachel! 2021-02-25 12:50:01 @DavidVidalJD Thank you! 2021-02-25 12:48:33 @EstateMatt Lol what if it attacks a person in self defense? Like Asimov's three laws lol 2021-02-24 21:51:11 RT @nikitaljohnson: “AI doesn’t work unless it works for everyone.” Hear more from @rajiinio in this interview. https://t.co/GQs50IPtsN #… 2021-02-24 18:44:04 @hypervisible Students surveilled at school while their parents are being surveilled at work 2021-02-24 18:41:28 RT @FortuneMagazine: Your data is a weapon that can help change corporate behavior https://t.co/c3KBS2rbFc 2021-02-24 12:59:33 RT @Combsthepoet: Tomorrow. Please join us. https://t.co/rouT3LkeaS 2021-02-23 17:15:36 RT @ruchowdh: In my last article as Parity founder - a very salient and much-needed discussion on the risks of growing the "algorithmic aud… 2021-02-23 13:33:26 It speaks to the acumen of BLM organizers I think, and how they understood the importance of prioritizing stories over stats, when it comes to explaining the impact of unjust death. 2021-02-23 13:30:11 There's a moment in this interview I keep thinking about. Where Natasha talks about George Floyd & 2021-02-23 12:55:30 @dhh @dafacto It would be great if this supported Markdown. If anything just to allow for users to add equations & 2021-02-23 11:49:33 RT @JaniceWyattRoss: Daughter 1 was taking an exam today being proctored by some type of software that apparently was not tested on dark sk… 2021-02-22 22:07:17 Whenyou measure include the measurer! https://t.co/rFuVPEe9Hu 2021-02-22 20:20:19 RT @BigDataMargins: Coming up this week! Big Data at the Margins is pleased to present: "Digital Policing: Facial Recognition Software & 2021-02-22 15:33:52 @mmitchell_ai Sending so much love Hope you are ok! 2021-02-22 15:29:28 lol English is such a disappointing language https://t.co/KG385DLBK1 2021-02-22 06:42:23 RT @mozilla: Technology has never been colorblind, but calling out racial inequities of data and algorithms means facing denials and backla… 2021-02-21 15:33:36 Closed questions with negative clauses are impossible to answer. "Is it never going to happen?" "Yes, it's never going to happen." = "No, it's never going to happen." "Are you not going?" "Yes, I'm not going" = "No, I'm not going." 2021-02-21 15:20:04 RT @techreview: Algorithms now decide your credit score, which patients receive medical care, which families get access to stable housing,… 2021-02-20 22:34:39 @red_abebe oh yeah @databoydg & 2021-02-20 22:32:10 @mutalenkonde Man, so sorry to hear that 2021-02-20 21:59:25 The fact is that there's currently almost nowhere that is 100% safe for Black people to do their work right now. That's incredibly upsetting - and a tragedy for the field. With Meg & 2021-02-20 21:59:24 I haven't been working for long, but have already witnessed a Black woman at a nominally civil society org being devalued (ie. literally paid $30k less than peers with less experience), then tossed out in an equally hostile & 2021-02-20 21:59:23 People need to understand that what's happening to @timnitGebru & 2021-02-20 21:21:22 Unbelievable how much of a difference one Black faculty makes - in her first year! Amazing. https://t.co/9jNdhQTEes 2021-02-20 00:37:24 RT @dmetaxak: Google recently added a feature allowing users to search for "Black-owned businesses near you". They're advertising it all ov… 2021-02-20 00:28:38 @alexhanna 2021-02-19 22:17:48 Seriously. @mmitchell_ai & Completely Google's loss. https://t.co/JEymKCLDcu 2021-02-19 22:14:55 @alexhanna Sending strength to you, @cephaloponderer & 2021-02-19 22:12:57 @mmitchell_ai So sorry 2021-02-19 22:12:36 https://t.co/gutoyvSsXK 2021-02-19 18:45:06 @BrownSarahM Yeah, I also wonder how this compares to the audit of medical devices actually (groups like BSI kind of act as government certified corporate watchdogs, still hired by companies to meet some externally & 2021-02-19 16:20:28 Whoo! This is unbelievably inspiring, given this lab's earlier commitment to do tech + society work without taking *any* money from corporations. https://t.co/2J4GpeTRO3 2021-02-19 14:40:02 "He tried not to panic. He wanted to make an appeal ... But nobody, not his teachers, not Ofqual, not government ministers, would say what counted as evidence for him to mount a protest. He had been told what he was worth and given no means to disagree." https://t.co/TH4FoE9wPe 2021-02-19 14:24:00 @Mehtabkn Sure, feel free to send over an email! I've also written about internal audit pros/cons here (https://t.co/UcQTUanEvZ) & 2021-02-19 13:43:47 RT @ndiakopoulos: Once you start digging the algorithms are *everywhere* in government ... here's how the Treasury Department calculates ta… 2021-02-19 01:36:49 And by new perspective I mean perspectives prioritizing other stakeholders, and not just the company's interests. Product risks literally look different when you value different people. Audits from an outside lens thus includes questions internal audits may not even think to ask. 2021-02-19 01:33:49 Though a completely neutral independent audit isn't really the goal. Even regulators have interests that compromise & 2021-02-19 01:29:23 This is an important point when it comes to auditing. I actually consider paid consultants (ie third party auditors paid by a company to complete an audit) as interchangeable in certain ways with internal audit teams. They are valuable in their own way but certainly not neutral. https://t.co/bhH2gRnWwI 2021-02-19 01:25:13 RT @Mehtabkn: Third party ≠ neutral ... 2021-02-19 00:01:22 RT @shalinikantayya: So incredibly thrilled to announce that #CodedBias has won three SIMA Awards for Best Director, Best Sound Design, and… 2021-02-19 00:00:00 RT @erichorvitz: Knowing when & 2021-02-18 23:35:53 @mmitchell_ai 2021-02-18 20:06:32 RT @mmitchell_ai: ...And this is how I find out. I'm so glad for all the trust they've rebuilt. It seems I've been completely erased and my… 2021-02-18 17:26:22 RT @alvarombedoya: Dear world, Dr. @TimnitGebru is a luminary. Her work has changed critical policy debates and will do so for decades to c… 2021-02-18 13:38:38 RT @FAccTConference: REMINDER - today is the LAST DAY for conference registration at a reduced price!! Don't miss this opportunity to sa… 2021-02-18 04:53:49 RT @hatr: Colleagues of mine analyzed A.I.-based job interviews. The software promises to be able to detect personality traits and be "fast… 2021-02-18 01:03:42 @anoushnajarian @SilverJacket @alexhanna @IasonGabriel @mmitchell_ai @timnitGebru @KatieShilton @black_in_ai @NewYorker @ruha9 @safiyanoble @jovialjoy @robotsmarts @tiffani @uwcse Thank you for bringing up this issue, though! I do think even contacting Pedro was a mistake, and a decision all journalists should continue to be seriously challenged on. 2021-02-18 01:02:50 @anoushnajarian @SilverJacket @alexhanna @IasonGabriel @mmitchell_ai @timnitGebru @KatieShilton @black_in_ai @NewYorker @ruha9 @safiyanoble @jovialjoy @robotsmarts @tiffani @uwcse Ah, I see. It seems @SilverJacket clarifies above that, in that tweet, he was really just tagging everyone he had contacted for the article, Pedro being amongst them. I also disagree with the decision to contact Pedro here, but don't think tagging him was intentional promotion. 2021-02-18 00:54:12 @timnitGebru @SilverJacket @anoushnajarian @alexhanna @IasonGabriel @mmitchell_ai @KatieShilton @black_in_ai @NewYorker @ruha9 @safiyanoble @jovialjoy @robotsmarts @tiffani @uwcse Yeah, I understand completely. Given all the active harassment he's engaged in and condoned, he should no longer be even approached as a valid source on this topic (or any for that matter). Sad to see him being continuously engaged with, despite his bad faith engagement. 2021-02-18 00:46:18 @SilverJacket @anoushnajarian @alexhanna @IasonGabriel @mmitchell_ai @timnitGebru @KatieShilton @black_in_ai @NewYorker @ruha9 @safiyanoble @jovialjoy @robotsmarts @tiffani @uwcse @anoushnajarian @timnitGebru fwiw I've read the article and Pedros is not mentioned by name, nor is he necessarily painted favorably. I totally understand the valid concerns here but just want to reassure you both that he wasn't legitimized or promoted in this particular piece. 2021-02-17 19:02:42 RT @FAccTConference: This year's ACM FAccT 2021 will be held virtually from March 3 to March 10, 2021 and registration is OPEN. Our early b… 2021-02-16 17:49:31 RT @garnettachieng: the tech + society resource list i was compiling is here it's not done yet, but so far it has: -open access books -co… 2021-02-16 16:52:17 RT @FAccTConference: ACM FAccT 2021 was slated to take place in Toronto, Canada. However, due to the ongoing COVID pandemic, it will be hel… 2021-02-15 22:10:37 RT @sarahookr: Yesterday, I ended up in a debate where the position was "algorithmic bias is a data problem". I thought this had already b… 2021-02-15 22:03:30 @mmitchell_ai @timnitGebru So upsetting. Sending 2021-02-15 22:02:49 I assure you, Meg is the absolute last person on earth to ever deserve any of this. She is still one of very few I would describe as a true ally, constantly and consistently sharing power to make opportunity for people of color and advocate for them. So incredibly upsetting. https://t.co/nfGXRxOAMM 2021-02-15 19:18:58 The role of the affected population in defining & 2021-02-15 19:18:57 One notable oversight I noticed is this - @alexhanna is not just more "woke" or "informed" than the voice-to-face researcher. As a trans woman, she's also more at risk of the harm being discussed. This is exactly why she sees that harm more clearly, and exactly why she speaks up. 2021-02-15 19:18:56 This @NewYorker article heavily features our #NBIAIR workshop @NeurIPSConf & The conversation about research ethics is an important one that the AI field certainly needs to have. https://t.co/oO6U4al88v 2021-02-15 16:06:32 RT @hannawallach: Finally got a chance to post a (near) transcript of the talk I gave at the "Navigating the Broader Impacts of AI Research… 2021-02-15 04:11:35 @JulianPosada0 I genuinely wonder how many industry researchers pay attention to GDPR when deciding on how to leverage internal data for their projects. I think many interpret this to default to articulating commercial use & 2021-02-15 04:05:42 @MarkSendak oh curious - care to elaborate here? 2021-02-15 04:05:10 And in case people are wondering, no, the DeepFace paper makes no comment on consent. At all. Granted it was a diff time with a diff level of awareness, but the sense of entitlement to the data uploaded to people's profiles on the platform... is a lot. https://t.co/Ez4VipYqXx 2021-02-15 04:00:23 Why don't we have clear norms around internal corporate data & 2021-02-15 04:00:22 This secret Google dataset reminds me of the internal data used by Facebook in 2014 to train DeepFace - the first deep learning model for facial recognition, that held SOTA on LfW for a while. That dataset was composed of 4.4 million labeled faces from actual Facebook profiles. https://t.co/DyjWXoG6pI 2021-02-14 13:00:59 RT @tdietterich: Every once in a while, something amazing comes out on @arxiv. Check out this cool new ML book by @mrtz and @beenwrekt . ht… 2021-02-14 03:57:09 @alexhanna Awful. Sending you all the love and strength you need right now. 2021-02-12 21:32:18 RT @sherrying: I’m proud to share https://t.co/RjHBOLsHzl, featuring political and economic solutions and ideas generated by CASBS’s networ… 2021-02-12 20:28:59 RT @TCIAMN: We did it, Minneapolis. The City Council just voted to ban facial recognition in our city — a huge victory for privacy and for… 2021-02-12 20:26:49 RT @suryamattu: Good example of why we invest the time in building tools that allow for persistent monitoring https://t.co/9QHn9fVxnG 2021-02-12 13:30:55 RT @Jeopardy: Can you do the math? @AJLUnited founder Joy Buolamwini (@jovialjoy) presents the MATH IN THE WORLD category, in partnership w… 2021-02-12 13:16:58 RT @Patterns_CP: For @WomeninScienceDay, we asked Advisory Board member Inioluwa Deborah Raji how to get girls interested in data science.… 2021-02-11 19:29:58 Yeah, what I had said was, "Recruiting already vulnerable people into a hostile environment is abuse." Why bring minorities into conditions where you know they will be disrespected or otherwise mistreated? Without offering protection, that serves as its own form of violence. https://t.co/AMrPwFNoTH 2021-02-11 18:46:02 RT @lawrennd: This is quite different to my experience. My breakthrough progress always occurred through inspiration. I needed time and… 2021-02-11 18:28:14 RT @lucy3_li: a little reprioritization https://t.co/GydAEqtc6m 2021-02-11 18:17:41 @mutalenkonde 2021-02-11 16:53:14 @lopalasi @mutalenkonde @daphnehk @Abebab The primary concern is how I can protect myself given the fact that I'm Black and that everyone always knows it. 2021-02-11 16:51:28 @lopalasi @mutalenkonde @daphnehk @Abebab As in the Nazis will always know we are Black, because we often look very different and can't easily hide this. And they already know which local businesses are black owned, already know where to find us. So identification/exposure as a Black person isn't often a primary concern. 2021-02-11 16:50:11 @lopalasi @mutalenkonde @daphnehk @Abebab Hm, this is something I hear a lot from Europeans, likely because the traits for which people experience discrimination there can be hidden (ie. immigration status, sexuality, religion, or ambiguous ethic groupings like being Jewish or Hispanic). IMO Blackness is always visible. 2021-02-11 16:38:53 @lopalasi @daphnehk @Abebab @mutalenkonde I don't think so, in this case. Community-led lists have the same highlighter effect & 2021-02-11 16:33:12 @mutalenkonde @daphnehk @lopalasi @Abebab lol I'm sure @daphnehk meant well. Though yeah, race issues are always much simpler than we'd like to pretend, I'm getting into the habit of pointing out when the solution is there and the thing to do is just acknowledge the situation, however uncomfortable & 2021-02-11 16:26:32 @daphnehk @lopalasi @Abebab @mutalenkonde + an inappropriate process can yield a list that doesn't embody the original cause of the effort or does so without transparency. If not done well, this could be an oversaturated list of any Black business vs. a mode to highlight undervalued gems in true need of further promotion 2021-02-11 16:16:04 @daphnehk @lopalasi @Abebab @mutalenkonde hm, I'm not sure if it's that complicated. It most definitely matters who made the list & 2021-02-11 15:19:27 RT @beenwrekt: I’m excited to share a new textbook @mrtz and I wrote: "Patterns, Predictions, and Actions: A Story about Machine Learning."… 2021-02-11 00:47:14 With all this talk of 90 hr "intense work" weeks, I'm feeling especially grateful for the positive role models I've miraculously surrounded myself with. People that prioritize rest, can say "No" & 2021-02-10 22:50:02 @jackclarkSF 2021-02-10 19:47:27 RT @mariadearteaga: This year's @FAccTConference offers: 1⃣Discounted registration for countries listed by ACM as economically developing (… 2021-02-10 19:46:20 "Discovering multi-billion dollar spending deltas across different info sources from same government" https://t.co/jCugLCmrGs 2021-02-10 19:44:42 RT @Patterns_CP: Patterns is proud to celebrate Black women in STEM, including Advisory Board member @rajiinio. Inioluwa Deborah Raji belie… 2021-02-10 16:52:33 RT @jovialjoy: #ad As part of @Olay’s #FaceTheSTEMGap mission, I’m making an appearance on @Jeopardy to raise awareness for the gender gap… 2021-02-10 13:58:01 RT @FAccTConference: Registration is now **OPEN** for ACM FAccT 2021! The conference will be held March 3-10. Early bird pricing is availa… 2021-02-10 04:19:09 RT @red_abebe: The video is up: https://t.co/tSattxKKcH It is well worth your time and I especially appreciate the discussion on time. T… 2021-02-09 16:19:50 RT @bobehayes: Webinar February 9, 2021 2021-02-08 18:19:10 RT @easears: From the wrongful arrest of a Black man based on a faulty facial recognition tool to the firing of @timnitGebru from Google, 2… 2021-02-08 18:18:20 "Beneath the veneer of new & Ironic when tech meant to push us forwards ends up entrenching past mistakes. https://t.co/Ptor0ZKW2i 2021-02-07 23:00:44 RT @mercola: “Arguing that you don't care about the right to privacy because you have nothing to hide is no different than saying you don't… 2021-02-07 19:00:08 RT @RaceNYU: Incredibly important new study from @rajiinio & 2021-02-07 00:26:25 @genmaicha____ (Retweet) 2021-02-06 16:49:50 Yes! And they should really learn about engineering responsibility too, including something we call... documentation. https://t.co/LrcSoRvfg8 2021-02-06 13:43:20 I still remember when my little sister once asked me if there was an African Disney princess and I had to say "Nala" (from Lion King), as if we don't have this rich history of actual human princesses with incredible, interesting stories. 2021-02-06 13:35:56 RT @UpFromTheCracks: I just sent my kid to school IRL for the first time this week and discovered they’re using facial recognition temperat… 2021-02-06 13:15:07 I applaud the push for diversity in #Bridgerton but can't help thinking - so many actual Black characters were actually around back then, there are dozens of great stories of Black royalty in Africa. Wish someone would finally write a series about them. 2021-02-06 12:19:32 RT @_KarenHao: As @rajiinio says, “You just can’t keep track of a million faces. After a certain point, you can’t even pretend that you hav… 2021-02-05 23:37:32 RT @mmitchell_ai: I am concerned about @timnitGebru 's firing from Google and its relationship to sexism and discrimination. I wanted to sh… 2021-02-05 20:37:59 RT @jennifer_e_lee: Y'all, SB 5116, a bill that would ban discrimination via automated decision-making systems has just PASSED out of the… 2021-02-05 20:35:47 RT @dsango: “We were shocked by the outcomes of the research/project/etc.” The dataset: https://t.co/JkpLuGzkGs 2021-02-05 14:22:46 RT @robotsmarts: Facial recognition - how did we ever get here? When most of us researchers started in this domain - it was an interesting… 2021-02-05 14:13:12 Also, something that doesn't quite make it into the article is how facial rec was motivated by a certain type of surveillance problem from the beginning. It's unclear how much, in that context, consent was ever to be prioritized in applications anyways. https://t.co/7352rtZGfG 2021-02-05 14:07:33 @neilturkewitz @hypervisible @_KarenHao @techreview Thanks for sharing this! Will check it out. 2021-02-05 14:06:35 The fact that we fall into this trend of data hoarding *even when handling the most sensitive biometric info* reveals something about our priorities as a field and how it's shifted away from recognizing the humanity of the people present in our accumulating datasets. https://t.co/HHxoPwZedS 2021-02-05 14:06:34 In this article, @_KarenHao summarizes our paper so concisely. Facial recognition is just the latest (& https://t.co/w2qTe2WWyy 2021-02-05 13:46:30 RT @ambaonadventure: https://t.co/BRK58Sl6OK Read @_KarenHao's excellent summary of @rajiinio and @genmaicha____ important new work. (1/3) 2021-02-05 13:46:15 @nunuska @kelarini Wow, that was lovely. Thanks for sharing! 2021-02-05 05:29:44 @kelarini This is honestly what hurts me the most 2021-02-05 01:19:35 @alexhanna lol clearly all the judges watch the same TV 2021-02-05 01:17:43 @alexhanna I know I shouldn't care but man, it hurts. 2021-02-05 01:16:21 ...how is there not a *single* woman of color on this list? #GoldenGlobes https://t.co/slvlfkXE5O 2021-02-04 23:41:22 RT @melindagates: Pass this along to a student in your life who should hear from @serenawilliams, @YaraShahidi, @rajiinio and more that com… 2021-02-04 20:35:51 RT @lauramoy: Hi friends! Last week on my birthday (!), Illinois Law Review published my first law review article, "A Taxonomy of Police Te… 2021-02-04 13:10:03 @hildeweerts @roeldobbe @Lauren_wbg oh my goodness, thank you! 2021-02-04 13:07:58 RT @ForbesTech: https://t.co/rA41F83y76 2021-02-04 12:29:20 @roeldobbe @hildeweerts @Lauren_wbg oh man, we really need to put this up on arvix! 2021-02-04 05:44:47 This is an interesting full circle moment - when working on outreach via @hashtag_include, we heavily relied on @codeorg resources to bring coding curriculum to local students in low income & Incredible to actually be able to contribute something back! 2021-02-04 05:34:50 Happy to see this video out in the world! It's never been a more important time to celebrate #BlackVoicesForCS https://t.co/mpK79HF9Bn 2021-02-04 00:17:08 RT @vineshgkannan: Yesterday was my last day at Google. I left because Google's mistreatment of @timnitGebru and @RealAbril crossed a perso… 2021-02-03 23:55:08 RT @PrivacyPrivee: News release: Clearview AI’s unlawful practices represented mass surveillance of Canadians, commissioners say https://t.… 2021-02-03 22:47:41 @mmitchell_ai LOL I love this. 2021-02-03 21:03:17 Ahaha, I love this thread! Pure joy Congrats! https://t.co/c49a4aMymE 2021-02-03 16:52:46 @databoydg Don't even get me started on that one 2021-02-03 16:38:25 @BrownSarahM 2021-02-03 14:12:44 Knives don't require the hoarding of my biometric information. Also, knives work. Can't stand this dual use argument when used to justify AI deployments that aren't even functional. Also annoyed with comparisons that ignore the tech's unique, data-specific ethical implications. https://t.co/Uq6Y7jBhrX 2021-02-03 01:18:04 RT @LeonYin: Want EZ data dictionaries/directories for README files? Here's a short script to turn a dictionary like, {"column": "descripti… 2021-02-02 23:01:35 @mdekstrand I don't even know how to start thinking about this lol 2021-02-02 22:35:57 @undersequoias Honestly? The scare quotes around "suspicious persons" - I don't think NYT would have stated that ironically 2021-02-02 22:28:43 @undersequoias lol at first I thought that was a quote from the article 2021-02-02 22:24:06 RT @samuelmcurtis: Great preprint: the segmentation of FRT development into periods is very informative. https://t.co/UhJM8kF1zj https://t.… 2021-02-02 22:23:33 I really hope people working on facial rec development & https://t.co/LjbXrVxKtU 2021-02-02 22:17:24 @jackclarkSF Thanks! We're hoping it can be a resource for those trying to make decisions about the technology and it's use, especially those thinking about audit design. 2021-02-02 22:14:57 Facial recognition was literally conceived with the task of matching mugshot photos in a book of "suspects" & 2021-02-02 21:26:45 The main lesson we learn is this: from the *very* beginning, facial recognition was designed to identify suspects with the goal of apprehension, whether in the context of law enforcement, intelligence or immigration. Current "benign" commercial uses are relatively new inventions. 2021-02-02 21:26:44 A long overdue pre-print is finally out today! Me & 2021-02-02 19:57:18 RT @JuliaAngwin: So true. Tech coverage has become a civil rights beat. https://t.co/zNqeaKubdp 2021-02-02 12:07:32 @Abebab There's a good critique paper from our ICML workshop (https://t.co/FLeJhnI2ag) - one of my fave contributions, "Participation is Not a Design Fix for Machine Learning". They also wrote on op-ed on it: https://t.co/AS035TJgLo 2021-02-01 22:53:39 @databoydg @wgrathwohl lol this is blackmail... 2021-02-01 21:01:45 @alexhanna @megfenway LOL the only thing I miss 2021-02-01 19:09:43 Wow - thanks for the shoutout! Happy #BlackHistoryMonth ! https://t.co/7jh4IrkPKG 2021-02-01 18:17:32 @mer__edith @mmitchell_ai 2021-02-01 18:11:05 Now, I've found out he's also gone after @mmitchell_ai as well People need to understand how absolutely exhausting it must be for these women to be facing this harassment, for weeks now. https://t.co/prLOjpu0eU 2021-02-01 01:03:34 @zehavoc @yoavgo @krismicinski You too! 2021-02-01 01:02:49 @zehavoc @yoavgo @krismicinski If it helps, I'm on Chrome and the "I'm feeling lucky" button is still there lol. @yoavgo I think you need to literally go to https://t.co/2myEhOAyX6 to see it! 2021-02-01 00:59:40 @zehavoc @yoavgo @krismicinski You can check out the livestream though, I think the panels and talks covered a lot of the main directions captured in the papers. 2021-02-01 00:59:05 @zehavoc @yoavgo @krismicinski Gah, unfortunately not yet. We were thinking of writing something but felt like ideas are still maturing and so we decided to hold off until we've run this a couple more times to figure out how people were actually defining "participatory ML". 2021-02-01 00:41:21 @yoavgo @krismicinski We actually organized a workshop at ICML last year trying to get at how to technically design ML systems in such a way, to give individuals the power to shape the perspective informing their predictions and the objectives of the model. You might enjoy this:https://t.co/FLeJhnI2ag 2021-02-01 00:39:26 @yoavgo @krismicinski ok think I'm starting to get what you're worried about and it's reasonable. I think my take is that a model explicitly encoding certain inclusive values is safest for me so I prefer that being the default. But ideally, I think we'd want something like https://t.co/Emy7Ehtdah 2021-02-01 00:32:33 @yoavgo @krismicinski But they are already pushing a uniform agenda into the world via their products? It's just not actively or consciously acknowledged. And they're allowed to do that I guess, as an org with certain values. If they chose to push something different & 2021-02-01 00:25:58 @yoavgo @krismicinski What's the concern / feared outcome here? Sorry, I don't get the risk of acting with intention about the perspective/biases one chooses to represent with their data or model. 2021-02-01 00:24:43 @yoavgo @krismicinski I don't think one can "remove" biases. There's no such thing as an "unbiased", "neutral", "objective" value-free dataset here. The default still encodes societal default biases (which tend to oppress certain groups), every perspective embedded in data includes some kind of bias. 2021-02-01 00:21:11 @yoavgo @krismicinski I think it's as simple as that. This isn't about who's view wins, but just recognizing the reality of bias in every perspective (including the nominally "default", "neutral" perspective) and understanding that we have the option to approach things with intention instead. 2021-02-01 00:19:44 @yoavgo @krismicinski No, I'm not really proposing anything. I'm saying the current biases are imposed perspectives that aren't really neutral. They represent a certain worldview, that's actually not objective. Those oppressed under that world view actually have the option of doing things differently. 2021-02-01 00:17:20 @yoavgo @yuvalmarton @krismicinski Yeah, that's allowed. Curious though - what's the advantage of keeping that definition unsaid, and keeping these biases implicit under the pretense of "neutrality"? 2021-02-01 00:15:32 @yoavgo @krismicinski Yeah, I think the AI ethics stance here is that "neutral" doesn't exist, so we might as well be explicit in what decisions we're making and what's going on. "Neutral" is really anchored to a bunch of social biases and other things that means it's not really objective at all. 2021-02-01 00:13:13 @yoavgo @yuvalmarton @krismicinski I think being explicit in deciding what's good is better than implicitly deciding what's good? 2021-02-01 00:11:37 @yoavgo @krismicinski The proposal of alternative biases is there to remind people that the current, default biases aren't the only ones we have to accept. We can choose differently, especially if we claim to have values that prioritize inclusion, and the protection of people, etc. 2021-02-01 00:09:34 @yoavgo @krismicinski It's hard for me to assess that proposal without reading the paper. "Positive" could translate to "non-violent" or "less extreme", which would skew recs away from content that will radicalize ppl in harmful ways. Either way, this isn't the argument a lot of AI ethics is making. 2021-01-31 23:33:12 RT @conitzer: The AI, Ethics, and Society Conference's (@AIESConf) submission deadline is this Sunday (as long as it's still Sunday anywhe… 2021-01-31 23:31:03 @michaellavelle @yoavgo @krismicinski I agree with this completely - my perspective is actually that much of the harmful "biased" claims coming out of LMs are also just factually incorrect (ie. "All Muslims are ..." claims). Internet data is corrupt data and there are a lot of reasons to be skeptical. 2021-01-31 22:56:03 This guys has been targeting and harassing women on Twitter for weeks, including @timnitGebru & Exactly how many more times do we need to report him to get his account suspended? Ugh. https://t.co/PPzFUvCd7V 2021-01-31 22:54:12 @TaliaRinger This is disgusting. I'm so sorry! 2021-01-31 22:44:53 @yoavgo @krismicinski It's more of a comment of "look at how society treats this marginalized group & 2021-01-31 22:44:03 @yoavgo @krismicinski That's a reasonable concern. My understanding, though is that this is a question of intention more than the installation of one view over another. Right now, the biases we observe are being presented as some objective fact or truth. A lot of that work points out this is false. 2021-01-31 22:33:32 @yoavgo @krismicinski Either way, just because atm its not very actionable doesn't make the critique any less valid - that AI is a technology entwined with society itself (to a degree more intensely than other technologies that don't involve data), and that AI's issues are tied to society's issues. 2021-01-31 22:29:44 @yoavgo @krismicinski You mentioned it was coming from "non-tech" people. Sorry if I misunderstood! 2021-01-31 22:28:50 @yoavgo @krismicinski I'll admit though, that this kind of critique is often hard for technical researchers to grapple with, because it's an imprecise critique, and it can be really frustrating due to that impracticality. There are technical responses for what is being said, but that's still evolving 2021-01-31 22:25:10 @yoavgo @krismicinski I'll also challenge the idea that the critique is only coming from non-technical perspectives (it's mostly women & 2021-01-31 22:22:12 @yoavgo @krismicinski I don't know of a lot of AI Ethics work that's framed this way. I'd say their point is that society's issues (including racism) shape the AI systems - ie. AI doesn't originate from "nowhere" - and so its challenging to make AI less harmful without properly considering that origin 2021-01-31 02:32:01 @kchonyc lol I feel like they just listed the team's alma maters 2021-01-31 02:23:38 @JennaJrdn @hedgielib @KristinBriney @LibSkrat @elliewix @paigecmorgan @thecorkboard @jon_petters @ragamouf @Hao_and_Y @HannahGunderman @melissaekline @Dorris_Scott @PhDToothFAIRy Thank for putting this together - very helpful :) 2021-01-30 19:27:52 @LibSkrat Thanks for sharing - glad to check it out! I admit I'm really new to this idea of data librarians but quite curious. Any recommendations for where to start reading to learn more would be really helpful! 2021-01-30 18:41:08 @JennaJrdn @hedgielib @KristinBriney @LibSkrat @elliewix @paigecmorgan @thecorkboard @jon_petters @ragamouf @Hao_and_Y Wow, thanks so much for an incredible explaination! 2021-01-30 18:16:08 @JennaJrdn Hm was looking this up & 2021-01-30 18:11:01 Yes! Good reminder - @nycgov does a lot of things right with their Open Data initiative. I appreciate! https://t.co/flJQZOs9rd 2021-01-30 17:43:02 Wow - proof @themarkup is actually superior. https://t.co/kEoKHbL4Af 2021-01-30 17:29:23 @jackclarkSF @OECDinnovation Makes no sense - data is just as portable as functions, context is just as necessary. Also, commerical context is likely where this has a chance of getting addressed - public sector dataset variable names are rife with niche assumed knowledge that never gets articulated anywhere 2021-01-30 17:25:58 @jackclarkSF @OECDinnovation Hm, I honestly can't think of anything since it's not a widespread practice at all. Some github repos will have a data index in the README but that's it. Even eng teams that are super meticulous of defining function variable names get sloppy when it comes to data. Don't know why 2021-01-30 17:10:49 I call it a "data directory", some call it a "data dictionary". With every data release, someone will inevitably ask for one. It's almost never provided...because for the vast majority of datasets it doesn't exist. https://t.co/HdeJuAGHx9 2021-01-30 11:48:06 RT @jachiam0: Does anyone else feel a little spooked by the similarity between r/WSB culture and the 4chan far right radicalization pipelin… 2021-01-29 21:38:33 + congrats to @Aaroth & It's honestly really cool to see researchers working on these problems being recognized more broadly. An encouraging moment that indicates that we're on the right path! https://t.co/T5uEqZ5l6z 2021-01-29 21:38:32 Today I learnt @schock WON an Association of American Publishers (AAP) PROSE Award for Excellence in Physical Sciences and Mathematics for the book "Design Justice" So happy to see this work recognized!! @AJLUnited for the win (literally)! https://t.co/mvpqSxjcBl https://t.co/IBGYxnZM03 2021-01-29 20:38:03 @geomblog @KLdivergence @Aaron_Horowitz @mmitchell_ai @SwitchedOnPop what, how is there so much discussion about this random teen pop tune?? 2021-01-29 20:37:15 @KLdivergence @Aaron_Horowitz @mmitchell_ai lol you should! It's a compliment 2021-01-29 18:26:30 RT @_KarenHao: The latest image-generation algorithms are trained using unsupervised learning (unlabeled images), removing any possible bia… 2021-01-29 18:25:42 @mmitchell_ai lol I needed this! Yeah, trying to learn from @KLdivergence on how to seamlessly increase my shitposting & 2021-01-29 18:21:19 @mmitchell_ai 2021-01-29 18:00:25 RT @BigDataMargins: Big Data at the Margins is pleased to present: "Digital Policing: Facial Recognition Software & 2021-01-29 18:00:00 @mikarv @BigDataMargins lol I wish! Still looking for a non-ugly template :( 2021-01-29 17:53:45 @mikarv @BigDataMargins Perfectly captures the absolute drama of surveillance 2021-01-28 22:37:44 Wow. "As I watched a mob batter down the Capitol...I thought of my work in machine learning, thought of that drone strike, and wondered ‘did I unwittingly help create this?’" from "My friend radicalized. This made me rethink how I build AI" by @thejaan https://t.co/dEHdvtOyfu 2021-01-28 18:00:54 RT @_KarenHao: Ok AI Twitter hive mind! It's that time of year when @techreview starts researching our annual list of Innovators Under 35.… 2021-01-28 09:35:25 @kharijohnson @_KarenHao lol it's coming soon Will tag you once it's out! 2021-01-28 02:11:59 @nsthorat lol yeah, I actually agree here. So depressing though - the actual fate of people dependent of if those that do care can make interventions easy enough to implement by those that don't. ugh. 2021-01-28 02:10:23 @nsthorat But why are you building a car without understanding the importance of adding breaks? If you need someone else to build the breaks for you, then you really can't be trusted to build a vehicle that's safe. 2021-01-28 02:07:00 @nsthorat I agree that for ML in particular, eng responsibility is not always clear & 2021-01-28 02:04:07 @nsthorat Intent doesn't matter. If I built and sold a car with no brakes because I didn't feel like adding them, and thought I would get away with it, I'm still responsible for the harm that comes from that. 2021-01-28 01:50:09 The most tragic lesson I've learnt doing any kind of algorithmic auditing is that companies will not bother to make their product work, if they know the group it doesn't work for is powerless to stop them. 2021-01-27 23:45:44 About to talk to my laptop screen for an hour - if you're registered to watch me do so live, I'll see you soon! https://t.co/zmlg9h6j9G 2021-01-27 22:26:42 lol seriously! I even recently quoted his words in a paper. Him & 2021-01-27 14:03:47 RT @ndiakopoulos: This is an important critique of the limitations of for-hire algorithm auditing. Accountability only happens when there a… 2021-01-27 11:44:50 @alixtrot 2021-01-26 23:03:50 @aylin_cim @KendraSerra @ryanbsteed @FAccTConference This is my understanding of current norms as well. These norms are worth revisiting tho. It's already led to the slippery slope of IBM's Flickr face haul (https://t.co/Yvn4UNScHg) & https://t.co/7SjQsS5wI4 2021-01-26 22:15:15 @Charles_Butlerk @AOC This is gross and disrespectful. Please delete this or I'll report. 2021-01-26 21:59:38 @davedarko Good questions! It's definitely creepy - but it's a norm in computer vision to use any image that's public domain, especially the faces of public figures, with the assumption that well photographed politicians & 2021-01-26 21:21:37 Interesting thread. Fortunately, @timnitGebru & I hope ML researchers read it! https://t.co/5n4y59wmRr 2021-01-26 20:29:14 I'm also reminded of this - the infamous PULSE face depixelizer is another solid demo of bias in unsupervised image models. https://t.co/2B8ZlFNHsv 2021-01-26 19:55:53 Well, maybe I shouldn't be so surprised. @Abebab & https://t.co/9eXjtRGmEd 2021-01-26 19:25:37 Wow, the results here are genuinely alarming. Just like with auto-generated text, auto-generated images will encode harmful biases - here we have a cropped image of @AOC being completed with generated pixels depicting overtly sexual attire/nudity! https://t.co/OJHB01zU6m https://t.co/LSqu2SExzZ 2021-01-26 16:42:35 RT @echo_pbreyer: @simonilse In the US, the police bases operative decisions on AI algorithms, explains @rajiinio: "The police use of the… 2021-01-26 11:15:00 @Tarakiyee Oddly enough, I just saw this before seeing your tweet. GTP-2's anti-Muslim bias is definitely a thing! https://t.co/BepdydpVhg 2021-01-25 23:23:40 "Perhaps training language models on data from the open Web might be a fundamentally flawed approach." Yeah - I have never agreed with a conclusion more! 2021-01-25 23:23:00 Please read this amazing blog post by the authors that lays the risk out even more clearly - some pretty insane examples here, including GTP-2 sharing leaked/doxxed info, and copyrighted info in addition to personal data. link: https://t.co/gg1xHyriX7 https://t.co/l59RvKnkUS https://t.co/YlZV0hmdLk 2021-01-25 23:14:27 @adversariel @IreneSolaiman @Eric_Wallace_ @florian_tramer @mcjagielski That was an amazing read - thanks for sharing & 2021-01-25 22:57:04 @jabuppartyon Yeah, was very surprised the press did not pick it up lol 2021-01-25 22:56:25 @IreneSolaiman @adversariel Very spooky. Not sure where to file this mentally - in addition to bias, data corruption, environmental impact, consent, etc. there's clearly an infinite number of things to think about! 2021-01-25 22:28:49 An often ignored aspect of large language models is the security threat of input data leakage. It's no joke. It's not just Gmail auto-completing my home address - even models like GTP-2 trained on "public" Internet data will memorize sensitive info (ie. https://t.co/9bK1P1klnV) https://t.co/1YU1NJzDsf 2021-01-25 22:05:55 @morganklauss Well deserved! 2021-01-25 20:32:45 RT @ZoeSchiffer: More than 150 computer science educators have signed an open letter in support of Dr. Timnit Gebru. "We believe the comput… 2021-01-25 19:40:05 RT @benzevgreen: Reminder: two days left to apply for the @FAccTConference Doctoral Consortium! We welcome doctoral students in all discipl… 2021-01-25 14:23:47 RT @VidushiMarda: Today we're releasing a labor of love over the last year, "Emotional Entanglement: China’s emotion recognition market a… 2021-01-25 14:19:22 RT @feministandacc1: This Wednesday at 7 PM ET is our event with @rajiinio !!! For our full schedule and links to sign up for our free, ope… 2021-01-25 00:05:45 This is quite literally the thesis of our paper "Data and its (dis)contents: A survey of dataset development and use in machine learning research". https://t.co/23InWHyomr I hope ML researchers will take a look! https://t.co/NWfMjveByr 2021-01-24 23:11:04 RT @niedakh: Arguments made by @emilymbender & 2021-01-23 05:55:52 @nealkhosla @vc @sethbannon @zebulgar @winnerstakeall + debate on the reality of the "threat" of CCP & Ok this is my last message lol - gotta get out of these poor people's mentions! 2021-01-23 05:44:44 @nealkhosla @vc @sethbannon @zebulgar @winnerstakeall All the things I've communicated are part of the message of the original thread? And *you* may not be saying to have blind faith, but it's a position I see & 2021-01-22 23:55:14 @nealkhosla @vc @sethbannon @zebulgar @winnerstakeall That's given my values & 2021-01-22 23:51:04 @nealkhosla @vc @sethbannon @zebulgar @winnerstakeall I don't agree that the value of winning the race supercedes the risk. The reality of the situations I encounter with deployed AI systems is that rushing something out thoughtlessly ends up doing more harm than good, while also not really achieving the original goals for the tool. 2021-01-22 22:18:58 @nealkhosla @vc @sethbannon @zebulgar @winnerstakeall She doesn't go there in her thread but I've mentioned before that the cost is more than environmental (ie. https://t.co/zjlZQdA4e6). Either way, the gist of her call to caution isn't unwarranted. Being careful doesn't mean innovation has to stop, just that it's ok to slow down. 2021-01-22 22:13:19 RT @RaceNYU: Thanks to the work of people like @jovialjoy, @timnitGebru, @rajiinio, @GeorgetownCPT, we know the technology does not work fo… 2021-01-22 20:13:30 @vc @sethbannon @zebulgar @winnerstakeall I get that some will think the protection and consideration for those that may be harmed is not worth the dip in efficiency, but I think that caution in investment is more than justified, especially if these people are unlikely to experience the benefit anyways. 2021-01-22 20:09:52 @vc @sethbannon @zebulgar @winnerstakeall Her only point is that we can't run ahead and innovate without being careful. And we definitely can't justify any harm done based on just the promise of some aspirational and imagined benefit. The response to this should be an acknowledgment of slowing down, being more cautious 2021-01-22 20:07:41 @vc @sethbannon @zebulgar @winnerstakeall But also this isn't even really about who does it - the slowness of govt funding processes is a feature, not a bug, to push for the additional consideration of some often overlooked impact to doing certain kinds of research. We can import that caution into industry research too. 2021-01-22 20:06:09 @vc @sethbannon @zebulgar @winnerstakeall Yeah - why can't the government do it? We can tax corporations or the rich to fund it. The same researchers will be involved anyways, just with diff affiliations. 2021-01-22 19:53:47 @sethbannon @vc @zebulgar @winnerstakeall That interpretation is reasonable but really not what she's saying in the thread. She clarifies this in subsequent tweets: https://t.co/2FkLYppSrT 2021-01-22 19:26:22 @sethbannon @vc @zebulgar @winnerstakeall With my own comments, I guess I'm trying to clarify that the kind of equity she's calling for is more of a participatory & 2021-01-22 19:19:21 @sethbannon @vc @zebulgar @winnerstakeall The context is that Big tech cos are training large models & https://t.co/lJ7Xr36svA 2001-01-01 01:01:01