On Research Podcast – PRIM&R 2023: Insights and Takeaways

Margaret Rankovic: Welcome to the special edition of On Research with CITI Program. This podcast is for educational purposes only. It’s not designed to provide legal advice or legal guidance. You should consult with your organization’s attorneys if you have questions or concerns about the relevant laws and regulations that may be discussed in this podcast. In addition, the views expressed are solely those of our guests.

Welcome, everyone. This is a special edition of the On Research with CITI Program. We’re going to discuss a recent conference we attended and share our personal reflections and observations. Stay tuned to hear about emerging hot topics and takeaways. We’ll also cover tips for getting the most out of your conference attendance. The conference we are recapping is PRIM&R’s Annual Ethics Conference. We hope you will enjoy this wrap-up whether you attended PRIM&R or not this year.

Let me introduce myself. I’m Margaret Rankovic, director of content and education at CITI Program and I’ll be the guest host of this podcast. I’ve been attending and speaking at this conference for several years and my background is in IRB. To start off, I’m going to introduce the panel. Let’s start with Bharat.

Bharat Krishna: Thank you, Margaret, Bharat Krishna. I am the managing director of the CITI Program. And it was wonderful being at PRIM&R in DC in 2023 after I believe four years of the conference being virtual. Great to be back in person. I do like to expand the acronym, so I think we should, Margaret, mention that PRIM&R stands for the Public Responsibility in Medicine and Research. I believe they’re celebrating their 50th year next year. Is that correct, Margaret?

Margaret Rankovic: Yes. Bharat, that is correct. PRIM&R was founded in 1974, the same year as the National Research Act. PRIM&R’s new director, Dr. Ivy Tillman, announced their 50th anniversary at her keynote and that the organization has celebrations and reflections planned to commemorate this anniversary.

Bharat Krishna: That’s great. So that’s a big milestone. So it was a very important conference and I’m very glad to be here on the panel.

Margaret Rankovic: Thank you Bharat. Let’s hear from our next panelist, Dr. Jaime Arango.

Dr. Jaime Arango: Hi Margaret. I’m Jamie Arango. I’m the Vice President of content at CITI Program and I certainly echo Bharat’s statements. I’d also note that as almost a 20-year veteran from our conferences, it was great to be back.

Margaret Rankovic: Great. Cid?

Cid Gomes: Hi. Thanks for having me, Cid Gomes, head of IT for CITI Program and this was my first time attending PRIM&R. It was an eyeopening experience in a few different ways. They covered a lot of interesting topics and I can’t wait to dive in.

Margaret Rankovic: Thank you. And we also have Gina.

Gina Sullivan: Hi, I’m Gina Sullivan. I’m the director of sales and marketing and it was a pleasure seeing everyone at our exhibit table this year.

Margaret Rankovic: Thank you. So the five of us are going to give our reflections and observations of this PRIM&R conference. To give some context, we attended the 2023 PRIM&R annual conference, which was held December 3rd through 6th in Washington, DC at the Washington Convention Center. PRIM&R’s focus is on ethical, responsible and high quality research. What’s unique about this conference is that it offers content to both animal care and use and human subjects research oversight professionals. And in some cases that’s separate sessions and tracks, but there’s also content crossover sessions that cover themes that span the research enterprise.

The audience for this conference includes Human Research Protection Program, HRPP, directors and staff, IBC and IACUC and IRB administrators, members, chairs, IOs and SOs, federal agency representatives, pharma and biotech industry personnel, legal counsel, research subject advocates, QA/QI personnel, compliance personnel, research integrity officers and others involved in research oversight.

With that, I guess let’s start our conversation here with what was a highlight of the conference for you? Whether that be a favorite session or informative conversation or a poster you’d like to consider as a highlight. Bharat, do you want to start?

Bharat Krishna: Sure. First of all, I agree with what Gina mentioned. It was great to be back in person. I think this was the first in-person PRIM&R since pre-COVID, so it’s been four years. Of course the virtual conference over the years has been nice as well, but it’s not the same thing meeting everybody in person. And it was great to see the record turnout. There was so many people.

I think one of the other changes that PRIM&R has done is to fold the human subjects and animal research audience together in a single conference. They used to run them as two separate conferences pre-COVID. And I think bringing that audience together in my view is actually one of the takeaways from here. I thought that was a very positive thing to bring everybody together in one place.

And there were actually some sessions and I thought that was actually one of the sessions and takeaways that was quite interesting. There was actually a session, a few sessions where they actually spoke about why do we treat in the institutions, why do we treat these silos and really have the IACUCs meet separately and the IRBs meet separately? And this was really bringing them together. That research is really a continuum and I think that was a key takeaway for me.

And I would be remiss to not mention the other big theme, which was AI. It was all over the conference as no surprise. It was great to hear Ivy Tillman, the new director of PRIM&R speak and set the vision and stage for PRIM&R going forward. She took over earlier this year, so it was great to meet her also on the sidelines and to hear her talk.

Another session I really enjoyed was by Heather Pierce, who is the acting chief scientific officer for the WMC. She had a really nice talk as well during the keynote.

Margaret Rankovic: Thank you, Bharat. When you said research is really a continuum and we don’t operate in silos, that resonated with me and is such a key point from this conference and applicable to our work in content development and training as well. And yes, we will discuss the hot topic of AI later in this podcast. Let’s hear from Jaime. Can you share a key takeaway from your experience at PRIM&R this year?

Dr. Jaime Arango: Sure, Margaret. One of the major highlights for me was that we were able to all be back together. The obvious takeaway or the big takeaway as well was that technology, particularly AI was a serious focus for attendees and for the sessions, many that covered different aspects of AI. I think for the community, the big challenge is really, and AI is demonstrating this, is to keep current with research and how research occurs and how it changes.

Margaret Rankovic: Jaime, that’s a really good point and I actually learned something at one of the sessions too. That challenge for IRBs and oversight committees too is not just reviewing the tech but making sure that the research is using the most current version of the tech. So for example, as the tech providers increase privacy and data protection features to their apps or to their technology, that’s one of the safeguards you can have in place as an IRB is making sure that the researchers are using the most up-to-date and current versions of this type of tech.

Dr. Jaime Arango: Yeah, I think you’re absolutely right, Margaret. I think those points also bring out another concern or area that sometimes doesn’t really get a lot of focus, which is what happens to the research. I think IRBs and research administrators are focused on making sure the research occurs ethically and follows the regulations, but what happens after we’ve collected all this data and all this information, what happens next?

Margaret Rankovic: Great. Thank you Jaime. And yes, I agree. I think data management and data sharing and issues related to data ownership and access are really critical, tagging onto your point about what happens to the data after the research.

Now Cid, as a systems director at CITI Program, I’m sure you attended sessions more related to tech. Can you share a key takeaway you had from the conference?

Cid Gomes: So from my perspective, one thing that sort of jumped out to me was the conversation around AI. You just alluded to that part. So this is a recurring theme. It’s sort of the topic du jour. I’ve seen it in other conferences and I saw it from a different perspective at this one.

The one session that comes to mind I think is the one about AI is not the boogeyman, which is kind of a funny title. But the title of the session was a little bit tongue in cheek. The speakers admitted to that. But to their credit, they tried to sort of give an abbreviated history of artificial intelligence and how we’ve gotten to close to where we are today. And I think it was Dr. Rosenfeld went to great length to sort of peel the covers back from how a neural network works to an audience that some were probably familiar with this but others weren’t. So he had a challenge in front of him and he was able to dive into that and I thought that was interesting.

If you don’t know neural networks, you basically start with the question and the answer and then train the neural network on how to arrive at the correct answer and you’ll have at different training sets and different validation sets. Without getting too technical, the point is I think Dr. Rosenfeld went to great lengths to say this is how this works, this is how this black box works. You don’t need to be intimidated by this. This is something that we can work with. This is something that we can detect patterns with, root out biases and hallucinations and overall improve the tool as we go forward.

But before long, there were audience members that were bringing up the Boeing 737 Max. And I think the newer versions of AI, the generative AI is really what is the boogeyman in the room. And Dr. Rosenfeld and some of the other speakers that sort of acknowledged that that’s really the kind of AI that we’re more concerned with. And the challenge ahead of us is how do we build trust in operations like that? How do we buy the transparency that we need into those systems?

So I know that the EU just passed some regulations on this, so this is a hot topic right now. Salesforce, to their credit, have built a trust model around this approach to try and peel back those layers. So it’s something that I think we’re all sort of wrestling with and the research community is as well.

Margaret Rankovic: Cid, that’s really interesting to hear your take on the conference with the inclusion of the AI topic. Whenever I heard AI, I also heard it followed by ChatGPT. That seemed to be almost synonymous in some people’s presentations that they were discussing both AI and ChatGPT. And it was interesting to me to hear how the issues in ChatGPT have really emerged in the past year. Because I think a year ago we didn’t have any sessions that referenced ChatGPT and now probably five or six of the ones I went to were talking about both the use of it in research. How it’s being used to generate consent forms and make it more understandable. Using it to research something as a tool. Understanding when it is being used as the object of the research or the subject of the research. And then also issues we should be aware as research oversight as far as privacy concerns, bias, data sets, data sharing, proprietary use and such. So I thought that was really interesting and emerging topic that PRIM&R captured well.

Gina, did you want to share any key highlights from the conference for yourself?

Gina Sullivan: Sure, so for myself, my main role was to be at the exhibit table where I am meeting and greeting people. The highlights for me is that I’m meeting students who are brand new in their career just starting in research all the way through people who are very senior in their research careers or even getting ready to retire. So to be back in person and meeting and greeting everyone. Saying hello and obviously describing to them some of the products that can help them advance in their career, it was very special, especially not having been in person for so long. Just saying hello and meeting people I had spoken to over Zoom if you will, or over the 13-inch laptop screen.

One woman came up to me and said, “Hi Gina. I’m so and so.” I’m like, “Oh my goodness, I didn’t recognize you out of the 13-inch laptop screen. So nice to meet you.”

Margaret Rankovic: I know it’s almost like we need to have our virtual backgrounds behind us when we go around sometimes. I would definitely echo that and say for me, one of the big highlights of the conference, it was just being back in person again, meeting with colleagues, seeing people presenting in person. I think the in-person element was really positive.

A session that I took a lot from I thought was the FDA workshop the first day. I’ve been to many FDA workshop about reviewing FDA regulated research before. What I thought was unique about this one is they really incorporated current FDA guidance and the case studies we got to work with in groups and kind of dive into a little deeper. So we got to talk about human cell therapy products, which there’s recent FDA guidance on. Real world data and really talk about the nuances there in this kind of research oversight space. I thought that was really interesting.

I’ve heard all of us conference veterans really echo the sentiment that we enjoyed being back in person. And we’ve also referenced PRIM&R has a new executive director, Dr. Ivy Tillman. I did like meeting her and seeing her both as a keynote speaker welcoming us to the conference but also in a more intimate setting. I went to one of her sessions, she presented on professional development and team building in the HRPP. And it was great to see her in that role speaking more as a peer like, “Hey, this is what I’ve done with my team. These are the mistakes I’ve made, these are the challenges I’ve overcome.” And it was great to see her in that light as well as when she’s up giving a keynote speaking for PRIM&R itself.

Bharat Krishna: That’s a great point, Margaret. And I think that’s one of the big benefits of these conferences really is sharing things that we otherwise as professionals wouldn’t normally get access to, especially of certain leaders like that. Certainly very, very interesting.

Margaret Rankovic: And I did also want to say I thought the crossover content is unique for this conference as well. I went to a session on data sharing and they had somebody there from both the animal side and from the human subjects research side and they both got to bring different perspectives to the same issue here. So I thought that was unique, kind of bringing those together.

I presented a session about implicit bias in research and we were considered a crossover content session as well. We had a representation from an IACUC and also IRB. And by having those perspectives on there and giving a unified presentation, I thought that was a really unique thing about PRIM&R. We had about half and half, half IACUC folks and half IRB folks of the 70 or so people that attended our session.

Now that we’ve given our individual highlights and takeaways, the next thing I’d want to discuss with you guys is how do you plan to apply the knowledge or insights we gained from the conference to our work? What did we bring home from the conference?

Gina Sullivan: From my perspective, my main takeaway is the fact that the people at this conference and other conferences as well, they really need the products that we offer. We were introducing a brand new product called the IRB Protocol Review Course, but that course is one new offering that is a supplement to many of the other courses that we offer. So there’s the need to learn, whether it’s something like IRB protocol review or something that’s more about artificial intelligence, it is an evolving audience. It’s an evolving need for new courses because the industry continues to grow.

Cid Gomes: For me, it was important to meet a lot of the customers of CITI Program and understand how they’re using our learning management system. And just sort of A, get a sense for how they connect to us, how they pull data from us. What sort of wishlist items they might have for the learning management system for integrations going forward. Those are some of the key takeaways. So connecting with some of the vendors between sessions was pretty important for me.

Bharat Krishna: And I think those are all really great points. One of the things, these kinds of conferences, Margaret, I think there was over 150 sessions. One observation I made was almost a third, maybe more than that of the sessions had presenters that were also CITI Program contributors and authors, which I thought was great. And hopefully we’ll have more crossover there over time.

A couple other themes that I took away, which I think will be important for us as a training provider in the space going forward into 2024. A few other themes that I picked up on apart from the obvious AI theme that seemed to be everywhere. One was certainly around the importance of advancing equity and justice in research in sort of the oversight process. And I thought there was a lot of talk on that including by the FDA, I think the FDA in one of the sessions had their representative sort of underline the importance of the disproportionate impact of research sometimes on marginalized communities. And how important that is to sort of fix and get more participation in clinical research and clinical trials.

And I think that was a very important takeaway, especially in the context as we embrace new technology and get into more decentralized clinical trials coming out of the pandemic. With a greater broader acceptance of technology, hopefully we can bridge that. There are still other issues like social issues like trust, et cetera, with certain communities to embrace and get more people to participate in clinical research. I thought that was a very important topic.

There were also some sessions, some of them of course I couldn’t be at all of the sessions and I’m glad that PRIM&R records them so you can go back and watch them in case you were double booked and couldn’t be at a session. I believe there was one session which I missed in real time, which I hope to catch in the recording around the participation of new entities like CVS and Walgreens in clinical trials. And I think that’ll be very interesting to see how all that impacts training as more people come into the fold and how do we as a training provider really provide courses and material as a greater number of newcomers come into the field. That was one.

I think another thing that I would sort of say is important for the program I think is really just the takeaway, the continuing challenges and opportunities of conducting social, behavioral, educational research in the digital age. I mean using online platforms, social media, big data, collecting this data. Some of it is collected in a private context and private entities, but then federal research has all sorts of other rules. And how do you sort of bring all that together? What are the appropriate methods for data collection and data retention? Whether it’s health data or non-health data in this increasingly measured world and increasingly digital world. I thought that was a big sort of takeaway and for us to help train solve, find the expertise, share best practices. That’s another major takeaway.

Margaret, I’m sure you had some observations as well.

Margaret Rankovic: Yeah, absolutely. And it was interesting you pointed out the focus on justice and diversity, equity inclusion. I learned that there’s a new government agency called ARPA-H that I hadn’t been familiar with before and I found that to be interesting. That’s the Advanced Research Projects Agency for Health, ARPA-H.

Also, I guess this kind of segues into another question I wanted to throw out to the group. Is there is an opening panel. The question was what keeps you up at night? And they had representatives from different agencies kind of give what the trending topics are, what keeps them up at night.

The FDA, for example, the presenter shared, she was talking about organs on chips. It may work in function but we don’t know how organs talk to each other. The clinical trial diversity challenge, that was a big one she mentioned a few times. FDA’s ability to regulate things like cosmetics and personal care products. A new authority to get ingredients listed, enable toxicology work in science. So those are the things that she said would keep her up at night. So I just kind of wanted to throw it out to our panel here and see if there’s any topics we would add coming from this conference of things that would keep us up at night.

Bharat Krishna: Well, I’ll jump in with a couple. One I think for me is certainly as this is the accelerator and brake analogy, right? AI is there, it’s going to produce a whole bunch of innovation. We should certainly not slow down innovation. But there’s also many dangers that lurk ahead without proper testing, et cetera. So for us to find the balance I think between not slowing down innovation in producing great research, great products, other research that can help people not slowing it down while at the same time not making big mistakes along the way. And finding who’s accountable for mistakes that an AI makes on the road. Those are I think quite important balancing acts that we still need to solve. That one and then the list that the FDA person mentioned sounds like a pretty good one too.

Cid Gomes: I have a few things that keep me up at night, I’ll admit, and I think I’ve found some like-minded folks of one of the sessions late on Tuesday. Ann Maglia with UMass and Rhonda Ellepo, she’s in Texas, I believe. They ran a session called Building a Research Data Infrastructure. And the session, if you recall how some of those sessions were quite popular, some of them were near standing room only. This session was not so full and it perhaps should have been much fuller. That was one of my concerns because they were wrestling with some very big challenges on how to build out a proper data infrastructure to support what the researchers are requiring.

And the one thing that sort of stuck out to me is that there are researchers that are conducting research today and generating voluminous amounts of data. These are not gigabytes anymore. We’re talking terabytes of data and they’re not getting those terabytes of data on the infrastructure at the institutions where they work. And researchers are going to research and if you don’t give them the infrastructure they need to perform that research, they’re going to go outside of your environment. And by doing that, they’re going outside of your controls and that’s where you risk data leaks. You risk revealing private health information and you risk cybersecurity attacks because the data is moving off of your platforms where you’ve built the security off of your four virtual walls so to speak.

So there was a lot of concern that they weren’t able to get those resources and it was nearly unanimous. The biggest challenges that they were all sharing was finding the funding to be able to buy the infrastructure components they needed to build. So they had a good plan for what they wanted to build, but they were competing with other projects and other priorities internally at those institutions to try and get those dollars. And I don’t know if those challenges are properly illustrated to the staff that are making these decisions so that they understand the opportunity cost of not investing in this infrastructure and what sort of risks that could pose.

Margaret Rankovic: Cid, I think that is so key and I can definitely relate that your session probably wasn’t the most popular, the one you’re referencing, but it’s kind of like eating our vegetables. It’s something that everyone thinks we should be doing. We know we should be doing planning for data management, planning for data to sharing and making sure that mechanisms are in place and funding is in place, there are regulatory and policy requirements for this. But actually doing it, I’ve heard a lot of concerns about institutions struggling with the doing it part and securing funding and being able to enact those kind of plans and policies.

Bharat Krishna: One of the challenges in that is really you get funded for doing research on a specific project or a set of protocols, but data and data management and cybersecurity and research security are really infrastructure. They are the underpinnings across all the research you undertake. And so sometimes I guess that becomes a little bit of the problem where institutions have to solve for research infrastructure issues that aren’t necessarily tied to individual grants but really span across. And so those always, I think in this sort of a grant-funded sort of universe that is a harder one to invest sometimes.

Margaret Rankovic: There were some topics that weren’t covered at the conference that I would’ve expected as well. Topics that are I think kind of current in the IRB and HSR and HRPP world that we are discussing, but I didn’t see sessions on. Those include bots in research. I’ve heard a lot about bots in research messing up the research itself or interfering and how to manage and handle this, but I didn’t see any sessions on that.

I also didn’t see much on cell and biologics content and there’s a lot going on there with FDA guidance and developments and such. I didn’t see as much post COVID content as I would’ve expected. The past few years of PRIM&R, there’s always been content about how offices and organizations have responded to disasters and adapted and reviewed research differently and such. But I didn’t see anything this year on the COVID kind of direct content.

And finally, I’ve heard a lot about staffing shortages and team leadership and I didn’t see really any sessions on that as well. I guess for the rest of the panel, were there any sessions you were expecting to see at PRIM&R that maybe you didn’t see or topics not covered?

Bharat Krishna: I think that was a pretty good list there, Margaret. But there was a lot of content to give PRIM&R credit to cover the amount of change that’s going on. And coming back after four years, it’s hard to cover every topic and checkbox everything.

I would say cybersecurity I think is a topic that Cid mentioned and I might’ve missed it, but I didn’t see a lot of discussion on cybersecurity. I think research security is a key new topic. I didn’t see a whole bunch on that. I also don’t think I saw a lot of RCR type content. I thought it was a little bit maybe light as well. Especially with AI and ChatGPT, I thought honesty, plagiarism, the risk of not being original in research, those things were a little bit, maybe a little lighter. Or maybe I missed them in scanning the app and trying to pick interesting sessions out of so many.

I would say rather than what’s missing, there was one I found to be one of the oddballs of the sessions that for me at least attending was I went to the psychedelic research session, which I thought was quite interesting. And a few years ago cannabis research became more prominent and started showing up at PRIM&R and other venues. And we have developed some programs as the regulations have been changed in the different states. There’s definitely more activity in cannabis research and that’s been a hot topic over the last few years. And it feels like psychedelic research is kind of where cannabis research was a few years ago.

And that was actually quite well attended session. I was quite amazed at the number of people. There was a question to the audience, “How many of you are actively evaluating protocols or submissions for psychedelic research?” And quite a few hands went up and I thought that was interesting trend that I observed at the conference as well.

Margaret Rankovic: Great. I would also share to anyone who might be a first timer going to a PRIM&R conference or another conference, the presenters here are generous or thoughtful and appreciate questions as well. Myself as a presenter, I value when anyone asks a question. You get to expand on something you might have covered but you want to talk a little bit more about or clarify something or provide that follow up for later. So if you’re going to a conference and you have a question or a thought, please don’t hesitate. Please stand up and share it at the session.

Bharat Krishna: One other sort of tip I would give is at least I tend to do this and I find it valuable, is I do go through the posters. And you don’t have to sit and read every poster in detail, but if you can quickly glance at the main point of the poster and decide within a few seconds. I know your time is short or short in a conference like this, decide within the first minute or so if that’s a poster you want to really understand.

And scanning this helps you identify the handful of posters that folks have spent a lot of time coming up with possibly the previous year doing research on or years. It’s actually quite helpful. And these posters have to go through a process. They don’t just land up on the board there. And so they’ve already been filtered and they kind of reflect some of the more in-depth thinking that might be going on in the field that isn’t necessarily represented in the session. So I thought that was quite useful and I was also very pleased to see CITI Program mentioned a couple times in some of the posters. That was great.

Margaret Rankovic: I also did want to share that CITI Program, we did get the opportunity to meet our scholarship award winner at the booth. Jaime, did you want to speak to that?

Dr. Jaime Arango: Yeah, we got to meet Paul Braunschweiger Scholarship’s first recipient, Dr. Jennifer Bobenko. We awarded the scholarship the first year and unfortunately the pandemic sidelined the person the chance to meet, so it was great to finally meet her in person.

Margaret Rankovic: Great. Did anyone have any final thoughts they wanted to share?

Bharat Krishna: I was just going to say, Margaret, these conferences like PRIM&R are great for networking, catching up on what’s the latest in the field and certainly meeting some of your own colleagues. And as people are starting to work more remote, I think this is a great opportunity to physically reconnect. The social events are great.

And also as Gina suggested, get a sense of what the vendors are doing as well in terms of the exhibit halls, who are the players. A nice thing about a conference like PRIM&R is the regulators are there too, and it’s really great to hear from the FDA or NIH, et cetera. What are key top of mind issues for them? What’s keeping them up at night?

And I think we’re ending in a situation post pandemic with the best of both worlds. You get to attend a conference like this, go to these great sessions and then at least for a subset of them, if you miss the sessions, you can watch them virtually again afterwards in the recording. And I think that’s a great opportunity as well. It’s a great opportunity for continuous learning and it was overall a great conference. I look forward to seeing how PRIM&R evolves its model going forward.

Margaret Rankovic: Great. Thank you. Gina, you did mention some new courses that we’ve released and that people are interested at the conference. How can folks get in touch with you if they’re interested in CITI content or want to add new courses to their subscription?

Gina Sullivan: There are two ways. One would be email to sales@citprogram.org. Or we also have an 888-529-5929 number as well.

Margaret Rankovic: Thank you. Great. I would like to thank our panelists today for being on this podcast and for sharing their thoughts and insights about the PRIIM&R 2023 conference.

Thank you to our listeners for tuning in for today’s conversation. I hope you enjoyed our conference wrap up and got a few key takeaways. I invite you to visit citiprogram.org to learn more about our courses and webinars.