Participants 🔗
- David Giese - Partner
- Yujan Shrestha - Partner
- George Hattub - Senior Regulatory Affairs Project Manager
5 Key Takeaways 🔗
- PCCP as a Strategic Tool for AI-Enabled Devices: The PCCP provides a structured pathway for managing changes in AI-enabled devices, helping to address regulatory ambiguities between marketing submissions and notes to file. It facilitates continuous learning in AI models while minimizing submission burdens.
- FDA's Guidance and Practical Implications: The FDA's detailed examples in the PCCP guidance help clarify acceptable modifications and reduce ambiguity. It emphasizes the importance of crafting intended use statements to leverage PCCP effectively while maintaining compliance.
- Reduction in Regulatory Burden: Implementing PCCP can significantly decrease the workload for both manufacturers and the FDA by reducing the need for repetitive submissions for minor changes. This is especially useful in scenarios like retraining AI models or handling iterative updates.
- Industry Benefits and Risk Management: PCCP allows manufacturers to anticipate and document changes, providing a proactive approach to compliance. This reduces risks during audits and increases credibility during licensing or acquisition by larger companies.
- Future Prospects and Challenges: While PCCP offers clear advantages, its adoption requires thorough planning, precise documentation, and strategic alignment with development timelines. Industry webinars and FAQs are valuable resources to navigate its implementation.
Transcript 🔗
Opening Greetings and Technical Setup 🔗
George Hattub: Hi. Can you hear me?
Yujan Shrestha: Hey, George. Yeah, I can hear you. How’s it going?
George Hattub: Okay. Pretty good. How are you?
Yujan Shrestha: Good, good. Yeah. Thanks for coming.
George Hattub: No problem. Let me turn my video.
Yujan Shrestha: Yeah. Does the video work for you?
George Hattub: I'm not sure. I don't see it.
Yujan Shrestha: Can you give me yours? Yeah, I can go inside. You know, it might be video settings.
George Hattub: That's okay.
Yujan Shrestha: Yeah, I'll fix it next time. I don’t understand why it doesn't let you. Come on. Oh. What video panelists ON most on. I don't know if that.
George Hattub: Okay. No problem.
Yujan Shrestha: Let me insure that. David's supposed to be coming to see. All right. Let me turn on the stream on LinkedIn. All right. There we go. Going live broadcast has ended. That's not what I wanted to do. Okay. Go on. Hey, David. How's it going?
J. David Giese: Hey. Good morning. Hey, George. How’s it going?
Introduction of Panelists and Session Goals 🔗
George Hattub: Pretty good. How are you David?
J. David Giese: Hey, I'm doing great.
George Hattub: Excellent.
J. David Giese: Yeah, it's been a good week.
George Hattub: Great. Getting things done.
Yujan Shrestha: Yep. All right. It looks like we're are live. Yeah. Hope everyone's having a great week. We've been having a good week, too. Lots of interesting stuff going on. Definitely gearing up for the holidays. So yeah, today we've got, David. We've also got George on the call. George has been one of our talented regulatory experts. He's been in the industry for 30+ years. And we're happy to have him on the team. So today's topic is a really interesting one. It's the predetermined change control program guidance that recently got finalized. The guidance is specifically on the topic of AI enabled medical devices. However, I think we can learn a couple of things from this. To apply the PCCP even more broadly to that. So I figured, you know, today let's just start out with just kind of talking about maybe some high points and kind of what we have, what are big takeaways from like the PCCP Guidance are I can attempt to go first at this. I think the PCCP was meant to fill in a gap where there was this always the question of, you know, should it be a new marketing submission or should it be a note to file what to do in the middle?
Overview of PCCP Guidance for AI-Enabled Devices 🔗
Yujan Shrestha: I think the PCCP aims to add some more detail, or at least give us another tool on that pathway. And it bridges the gap between the between the two worlds. And I like how the FDA gave a lot of examples on what they consider going into a PCCP and what kind of changes would be acceptable in a PCCP or not. I'm looking forward to using this new tool in our upcoming devices and for our existing clients and devices as well to de-risk the changes for the current devices anyway. Like we'll get more into the details as we talk about it. David, any particular takeaways? And George, I'd love to hear your about your takeaways as well.
J. David Giese: Yeah. So I would just say yeah, this was the statute that was passed in 2022. So pretty recently, I believe. And it's not it's interesting when you go to the actual statute, it's not that long. It's only like 2 or 3 paragraphs, but it it has some pretty big implications. And I think there's only really two small regulations that come out of it. And then the guidance on the other hand, is like or the AI guidance was like 50 pages long. And then there's the draft general guidance. I mean, it's a little bit shorter. So it's always interesting just how much, you know, like there's like two paragraphs of the law, like a couple regulations. Usually regulations are longer. And then there's huge. It's like all the stuff in the guidance. But yeah, I think it seems to me that it was really motivated by the desire for kind of continuous learning and AI models, and that was maybe what prompted Congress to say, hey, let's add this new pathway to market for device changes. But then, yeah, it seems like it's being used in a lot of ways. And I get the feeling the FDA is really excited about decreasing the burden on them for kind of boring type submissions like one example is I think I forget the exact name of the company that used the PCCP, but they wanted to support like different like temperature monitoring watches, I believe for like fertility prediction app. And previously they had been doing new submissions for like each new watch which I'm sure for FDA is like not opt like it's just a lot of work for and you're probably verifying it almost the same way. So I think they did too. And then for the third one, they did a PCCP saying, hey, for any new watch that comes on the market that we support, this is how we're gonna test it.
Key Takeaways from the PCCP Guidance Document 🔗
J. David Giese: And then now they don't have to go back to FDA over and over for this change. And I think with the antibiotic susceptibility testing, there's a specific guidance on that PCCP for that as well. I think because the same thing, the FDA is just dealing with so many submissions, which were, you know, like, I think the way you test it is always the same. And but it is changing the intended use because the that the drug bug combinations are going into the intended use statements. So there's not really a way around it. And I think so, that's my interpretation is, just to summarize AI kind of enabled it, but then now FDA is like, hey guys, let's use this for all these annoying situations that we can avoid having to do so many submissions for. But I don't know, I'm maybe reading too much into it, but that's kind of like my take.
George Hattub: Sure. And the product that I've been managing, we've done nine FDA submissions, four traditionals and five specials. And right now, I have three pending: two specials, one traditional. And so most of the specials are for changes being made to the algorithm and the FDA, it's very time consuming, even though the specials very seldom do they get cleared in 30 days, it usually extends. They don't get converted into traditional. But I think this supports your belief that it's going to reduce the workload to the FDA. And one thing you have to avoid is to make sure that your intended use statement is either generic enough so that you can utilize the tool. Otherwise, like you said, you're going to force yourself into having to do 510(k)s So that's how I look at it and how it can benefit companies.
Yujan Shrestha: Yeah, I guess my takeaway on like can you change your device intended use the PCCP? I thought my takeaway was that no, you couldn't.
J. David Giese: Yeah, I think I misspoke there like I think yeah, it's not changing the intended use but it's a change that would definitely require another submission previously. So yeah. No I agree.
Yujan Shrestha: So yeah. But like the use case of like the antibiotic acceptability testing that does seem like it changes the intended use or maybe like as George mentioned, maybe the intended use was crafted in a more generic way. So it actually doesn't.
J. David Giese: And I would say, just to be clear, we work on IBD’s software, but we ourselves are not IBD experts. So this is my kind of reading in from the sides, but let me pull up the guidance for quick and I can I can anyway. I don't want to derail the conversation, but I'll give you an update in a minute.
Discussion on FDA Submissions and AI Models 🔗
Yujan Shrestha: Sounds good. So I think that we should talk about some of the examples that FDA gave us. So I'm going to share my screen here real quick. I like to read these guidance documents in reverse. It's the most effective way for me to do it just because the examples give me something concrete to look at. And it also lets me calibrate it whenever FDA says something like significant or, you know, large change or whatever, like they're these words that leave a lot open to interpretation. And I find that the easiest way to calibrate myself on FDA's thresholds for that is to read these examples first, so read the examples first, and then I read the rest of the guidance document. So yeah. So these examples are pretty interesting. The first one is about this patient monitoring software which is this separate box that can go into like a hospital room. So this box can read various signals such as blood pressure, pulse oximetry, EKGs and a whole host of other signals. And then it can detect the onset of physiologic instability. And then when it does detect, it releases a audible alarm on that. And in this situation, the manufacturer has submitted a PCCP. They would like to retrain the model to reduce the false alarm rate while maintaining sensitivity.
Any thoughts on this modification? My question to you guys. Before this PCCP guidance came out, where would you have put this change? Like would you have put this as a letter to file? Or would you have said, oh, this is more of a special or a new 510(k), which one you think this would have been?
George Hattub: Definitely a 510(k) in my opinion, because you're talking about changing the algorithm. And with the FDA software modification guidance, I think that it's one of the thresholds to trigger 510(k). So that would be my vote.
J. David Giese: Yeah, see, I would have probably put it more under a letter to file. Just I mean it's one of those things where like just from an engineering perspective, if you're validating it with the same test data set and you're expecting that the performance to go up, I wouldn't see it as a very high risk thing. Although I'll be honest, George, your comment about the software change to an existing device guidance like I'm not, I would say super familiar with the workflow in that in like what examples they have. So yeah, I'd be curious to maybe dig into that.
George Hattub: Well, yeah, the reason why I would say it. So, we've all talked about the modifications guidance you can really, you know, expand that as far as your what your decision. But in this case talk about reducing the false alarms. So that would be feedback from the field. I would think that would probably be a complaint. And then you have to do a CAPPA. You're going to a quality system. And then once again, when the FDA comes to inspect you, the person has to read the letter to file and will be, you know, investigator. And so that would be my concern, that if this was let's say there wasn't a complaint involved, maybe it was just feedback or, you know, you noticed this in demonstrations. If you hadn't released the software and you had control over it, then sure, a letter to file would make sense. But it's in the context. And I think that sentence reduced the false alarms. You know, I’d be concerned about that, what would initiate the change and so that's why I would feel safer with a 510(k). But once again it depends upon the circumstances.
J. David Giese: Can I share my screen really quick? Yujan?
Yujan Shrestha: Oh, sure.
J. David Giese: Okay. By the way, they have construction going on in the background, so I apologize for any background noise. So George, first of all, I would say like it may not always be in response to CAPPA for something like this. Like I would think like a lot of times the clients are just always trying to make the model better. Even if there aren't any particular complaints, they're just going to be trying to make it better. Also is a real quick aside, I did look up the anti susceptibility testing guidance and it's changing the breakpoints. So it's not adding new drug bug combinations. Which I think changing the breakpoints would have previously required another 510(k). Anyway just as an aside. So that's I think that resolves that inconsistency. But by going this is the software change guidance. And I think it's kind of comes down to this question here. Could that change like retraining the model significantly affect clinical functionality or performance that are directly associated with the intent to use the device? So I think I guess it's kind of do you think we're going this way or this way? Right.
Yujan Shrestha: It's all based on this significantly. Right? It's I think the examples FDA gives in this document gives us some hints on what they consider significant or not. And I think that's where I came to the conclusion that retraining is just a note to file is with the help of one of the examples in this document is where I originally came with that idea because yeah, like I would have put this kind of change also has just is just a letter of file. But yeah, to me it's like making a device more cyber secure by just increasing the version level. Like all of your software dependencies, if you just upgrade all of them, people patch software vulnerabilities and those kinds of things so that definitively does not need a new marketing submission.
FDA probably recognizes that if they require people to do a new marketing submission, that those kind of changes would probably just in practice, be less likely to happen, which is what I worry about. If we had recommended everyone going and we're required to get a new 510(k) for something like this, where it's just making the device better, maintaining sensitivity and reducing the false alarm rate, it's making it better and retraining the algorithm. It's not like re-architecting the whole thing. It's literally like pushing go again on the same code you had before, just using different data. So to me it was like, that's more in line with like a cybersecurity update. By the way, this is all just my personal opinion. In no way reflects FDA’s stamp of approval on this, but that was my original thought process.
Real-World Applications and Challenges in PCCP Implementation 🔗
J. David Giese: I certainly think in the case of cyber security, there's the time issue that like FDA recognizes, hey, like if there's a zero day probability, we want people to update these things as fast as possible. The problem, currently is that old devices aren't getting updated at all. And so I think FDA's like, hey, we better not add a barrier to doing this cybersecurity patch. And so they're very clear on that point that you can just update it. I think so… It does seem a little different maybe than this in that sense that there's not really a time factor associated with it. But I totally agree with what you're saying, that if you require retraining, any time you retrain the model, you got to do another 510(k), people are going to do a lot fewer retraining, right? They're just going to say, yeah, sensitivity is good enough. We don’t need…
Yujan Shrestha: Yeah, exactly. Maybe I'm overstepping and like determining what society wants. I mean, I have been American, I have been a US citizen for t minus four years of my life. So I don't know, I think I kind of understand. I would kind of venture to guess, like as a society we would rather have better devices and we would want to encourage retraining and making them better, just as we want to encourage making our devices more cyber secure. Even though doing all the cybersecurity patches, upgrading…that has a non-zero risk. Like sometimes software packages break when you upgrade them. Right? And like we've decided that we're going to favor upgrading for cyber security to where it might affect not necessary safety, but it could introduce another bug. So we've already kind of made that determination as a society. Like, hey, there are certain categories of changes that we want a fast track that may introduce other issues.
J. David Giese: I totally agree with that. But I think what society wants is like this kind of like vigilante FDA. In the day it's like, what is the law say? And what are the implications for our clients if we do this, like when you get audited, are we what's going to happen and how risky is it? But I totally agree with what you're saying though, that yeah.
Yujan Shrestha: In the audit, I'd be curious to hear what you guys have experienced. So far I've only experienced one FDA audit where we’ve used the retraining as a note to file and there were no issues flagged in that audit that were around this topic. But we retrained the algorithm many times and just a note to file, but it was not brought up. Now that could just be that the auditor didn't review that part or it could be something deeper. I don't know if you guys have any other experiences on audits.
J. David Giese: I do not have experience at this situation. I don't know, if George would.
George Hattub: Yeah. I have a lot.
J. David Giese: I was guessing you would have alot.
George Hattub: And the thing is and also I follow what happens with other companies. And I do agree with you that what society wants, I look at like probably the model for the FAA. Okay. In other words, when they discover a problem, they probably fix it right away and they probably work closely, you know, with the airline carriers. And this is why the safety record for flying is so good. Because of that, you know, we only hear about maybe the near misses or whatever, but I think the FAA model is probably something where we should probably go to. In other words, if you know you have a problem, get it fixex, document it, don't try to hide it and then learn the lessons. In terms of FDA inspections, what I find is the FDA an investigator inspector has a whole different mindset. Then let's say the scientific reviewer at FDA. And so when they come in to inspect you, they're looking for systematic problems with design control or whatever. How did this get through? And so I think the first, based on my experience, the first thing they check is your complaints. Then you campers. And then there's always a danger that if you didn't submit a 510(k), that suddenly things would stop. That happens more, I think, in the physical device world than the software. I think that nowadays everybody understands how software works and FDA’s knowledge is that you could release software with a bug in, a known anomaly, as long as the risk is low. So that's the only reason, going back to the FDA guidance document for the predetermine change controls, it's interesting that they lead off with what I would consider a customer complaint. But I'm assuming that if you go back to that, they said that this is a candidate that would fit for pre determined change control things that you could fix on your own. Is that correct, Yujan?
J. David Giese: Yeah. I'm sorry, Yujan I'll pass it back to you. Yeah.
George Hattub: So that example they're saying that this would be eligible for it.
Yujan Shrestha: I don't think actually this was due to a complaint in the field. I think this was just like the manufacturer knows that they're going to continue to improve the algorithm, and they've elected to do a PCCP with that knowledge in hand. Is this the way that I read that.
George Hattub: Okay, sure.
Yujan Shrestha: So like I think this is something that we need to resolve. It's like if our previous recommendation on this was note to file and now there's a PCCP, it's pretty clear here that this is a usage of a PCCP. What's not clear to me is, is FDA saying you need to now do this as a PCCP or the previous you're saying that the note to file is no longer valid because we have a PCCP? I'm not sure. Like I don't know if that if the question makes sense, but it you know, so like beforehand like the tool that we had were note to file and traditional or all of the say 510(k), all flavors of 510(k), right. And exactly like which way it went was kind of up to the manufacturer. We didn't really have any other tools, right. And so what we were saying is there's kind of this gray zone here retraining to maintain or improve effectiveness. And probably, I'll say, without changes to intended use, right. Pre PCCP our school of thought was note to file right. But post PCCP it really seems like FDA in this example they're saying this is the PCCP, right. Now, my question is you know is this first method still valid. Like sure we have another tool PCCP in, but do we have to necessarily muted it because we have the tool, doesn’t mean we need to use it right. But is FDA saying that we actually need to use it. That my question that I haven't really well been able to get a good answer reading the guidance for.
J. David Giese: It seems possible to me that just FDA had a more conservative stance all along and that they would have expected most retraining to be a 510(k) and I mean, and then become apparent with the guidance in the way they're talking about the examples.
Yujan Shrestha: Or the other alternative is that like there's just been a lot of bias, right. Like FDA is seeing a lot of people submitting a 510(k), but they're not seeing that people that are doing note to file. I mean it wasn't a problem. All it's just a survivorship bias kind of thing. Who knows what it really is and maybe FDA thinking, well, rather than just definitively telling people, oh, there's a note file which that's going to be abused. Almost certainly. Right. Like maybe rather than doing that, the only other option is to include a new tool. Yeah, that could be another scenario as well.
George Hattub: I think it's interesting too on the E-Star. There's not a section for this. In other words, the FDA hasn't defined how to package E-Star other than say, put it into your device description or something and how do you handle that? Do you say that releasing something that maybe may have issues or we want to improve it but we just don't know just yet how it's going to happen. And how does FDA look at that if you're open and saying that things may change. In other words, you made a 510(k) just for a PCCP. And we've talked about it. If you select special 510(k), it asks the question, you know, is this for a PCCP and they warn you can't do it on a special. You know, you have to do it in a traditional I guess with FDA has to look at everything and then decide.
Strategies for Regulatory Compliance and Audits 🔗
J. David Giese: Yeah, you know, I think in fact, I'd say they suggest I forgot which guidance I read this in, but I think they suggest actually do it in a whole 510(k) just for the PCCP.
George Hattub: Just for that.
J. David Giese: The one time that I’ve seen that the PCCP was not separate, it was kind of an add on thing. But I mean, I do think a few of the examples that I've seen, I went through a list of a number of submissions that had a PCCP in it because it's in the 510(k) a letter. So you can actually go and see and I do believe several of them were standalone 510(k)s with all it was just the PCCP was the whole thing.
George Hattub: That would be the safe route. I wouldn't want to have my first 510(k) and then have it held up because you know the PCCP. So you're saying that yeah, you'd recommend just doing the 510(k) for a PCCP and have the FDA review it based on its merits.
J. David Giese: I would say yeah it depends on the scenario. Like in the case where let's say, you did your first 510(k) and then you added another hardware platform or you adjusted the break points. And you're starting to say, hey, this is this is getting to be really burdensome doing all these 510(k)s. Now let's make a PCCP so we don't have to be moving forward. In that case probably just doing the PCCP before it makes sense. But I think another strategy that we haven't used, but that I do think is this one we're going to try pretty soon here or maybe we have one we've already done, it is having a PCCP where it's kind of like the next batch of software changes, not for an AI retraining, but for other software changes. And I think in that case, it wouldn't really be helpful to do it as if you need a whole 510(k) just for that, it's not really that useful.
George Hattub: Yeah. So I think the FDA is going to have a webinar in January on this given, I think that's going to be their way of explaining how this works. Perhaps. I saw that.
Yujan Shrestha: Hey, have you ever done an abbreviated 510(k).
George Hattub: No.
J. David Giese: No, I've never done that.
George Hattub: Yeah, I think it abbreviated one. You have to have the declarations of conformity and I've never seen any as far as special 510(k)s I think that I've always had the belief do the under-utilized fast track.
Yujan Shrestha: Yeah.
George Hattub: Like the last time we took care.
Yujan Shrestha: I mean, not here though. Like you can't use a special or it's not listed to use special to add a PCCP.
George Hattub: But within the abbreviated you could. Yeah.
Yujan Shrestha: Yeah. That's what they're saying here is if you wanted to add to establish a PCCP for device prior to manufacturer implementing I guess this is not actually quite they if your device is already on the market actually. Sorry I think that might not be completely accurate. What my statement is. Yeah I'm not sure actually. Can you use a special to add a PCCP to a device already on the market for a 510(k) cleared or a De Novo cleared device? I don't know the answer to that. I don't think this this tells us actually. But I mean, safe to say that you could definitely do it as a traditional 510(k). And that has been done. There's a couple of examples of that where I think all that they did is add a PCCP to it. I also mentioned that there's a couple of scenarios that we're thinking about using a PCCP, right?
I think there is the premarket submission. I think if you have known changes, you can speed up your pre-market submission, f you know the changes you're going to make ahead of time, perhaps adding it as a PCCP could get you a pre-market clearance, or otherwise you would have had to wait for those changes to be implemented. I think the other place that it may be possible, like what if you got some findings from FDA during the doing the whole letter? Could you add a PCCP? I might be too late to add a PCCP in that point, right?
J. David Giese: I think they state that they wouldn't allow you to add a PCCP during the submission that you'd have to resubmit.
Yujan Shrestha: Okay. So it really is like if you could predict ahead of time what the changes would be. Now the PCCP potentially speed up that premarket submission timeline. Seems like FDA have given us like it used to be two things to worry about. And now there's three as well. Now it's not a dilemma anymore that the trilemma right.
J. David Giese: Well, maybe it's we have more options to strategize about. And I do think especially with MedTech OS in the way that we can kind of generate documents from the notion databases pretty easily. I think it's really nice, if we're using notion for software development, we have the next several milestones kind of mapped out in our product development roadmap. And if the last two milestones, we don't get there by the time we want to submit, it's relatively easy. If we have all the requirements, the verification protocols, the software risk assessments done for those features, now we can have a flag where we just say, okay, let's now generate a PCCP for these and it should be almost turnkey within MedTech OS. And that's why I think, you know, anyone could use this. But especially with tooling like MedTech OS, it's could be very convenient and straightforward and I kind of think FDA go for it from my I think I talked to Jessica, forget her last name, she was one of the authors on the PCCP guidance last December. And I specifically asked her about this, and she said that she thought that was a valid way to use a PCCP. I think Yujan you said you had a similar interaction with maybe someone else.
Yujan Shrestha: Yeah.
J. David Giese: And anyway, I think for software that that's a really interesting round and I haven't seen any submissions where that's, it seems like that's been done and we haven't yet actually done it ourselves. But I just think it's a great option that that could really optimize the software development roadmap. I don't know, is it clear like what I'm talking about though, to all like how that would work.
Yujan Shrestha: Yeah, it is. And I'll come back to the question is the PCCP in that case a requirement or is the old note to file method. Because typically like the changes that we talked about there were like we're going to add like MDDS stuff. We're going to add kind of lower risk items, right.
J. David Giese: No, I'm talking about for things that would not. Yeah. If it's like that then you just don't even bother. Just do it. Do it as a note to file but I'm talking about specifically things that would have required a 510(k) or most likely would have.
Yujan Shrestha: Most likely would have. Yeah. Well most likely would not have been note to file. Okay. So what we're saying is like I mean like there's still a little bit of fuzziness there, like, even those changes that are of course changes that definitely would be known to file. There's changes that definitely would be 510(k) that would go on a PCCP, but there's still an overlap. And like we would still have I guess it's still a dilemma now in this case is that note to file or PCCP. You still have to make that make that call. Whereas before it was just a note to file or 510(k) and there's no like middle option.
J. David Giese: Yeah I totally…it's that significant spectrum like where do you land. Is this significant? Is it really significant? Maybe it's not quite significant enough to be really you know like and down with there's like that option in the middle for the PCCP and it kind of depends on the risk tolerance of the company I think is probably all of us as like the user base gets bigger for bigger companies. The risk benefit trade off shifts, because now it's like, hey, if FDA gives us pulls it off the market in an audit, that's now a massive problem and we're losing tons of money from it. So the relative trade off of, hey, let's just do a 510(k), it starts to shift. Whereas for the startups where they're like, hey, we don't have anyone using the product anyway, even if they took this off the market.
Audience Q&A on PCCP Use Cases 🔗
J. David Giese: Like it stinks. But we just want to get to a feature full enough product that people will buy it, which we don't even have that. Then they're going, their risk tolerance is going to be more shifted the other way. And anyway, so I think, yeah, I, I totally agree, it's still blurry, but it's at least nice to have this option in the, you know.
George Hattub: So, the example I was mentioning earlier, where with this one product, we've submitted nine, 510(k) and four of them been traditional five of them special. The reason we went to that was because we didn't do a special. We did a letter to file. And then when we followed up two years later with the traditional, the FDA asked us how did this feature get cleared? And we were like, letter to file. And, you know, it was a stressful time because they weren't comfortable with that. And we had to supply all the data during that traditional 510(k) to kind of cover that. And eventually we received clearance. And that particular feature was covered. And from there the client was risk adverse. You know, they learned the hard way. We're oh, okay, any change, we're going to do a special because we never want to have to go through that again. So we didn't have, you know, the this tool that we're talking about right now. But that's one of the reasons why we have so many specials and like it's so great. Now we have five cleared specials and we have two pending for the same software and I guess if this option was available way back when, this would have made a lot of sense to the client. Because, you know, when you start talking about all the work that goes into, you know, seven special 510(k)s, significant amount of time.
J. David Giese: That's really interesting. So, it's not just not just being caught on the audit side where they're less technical generally and it may not necessarily but catch on to it. But also like on the follow on submissions that can creates problems where they're like hey.
George Hattub: Yeah yeah. That and the other way you can get burnt is that your competitor sees this feature and then they submit a 510(k) and then, you know, the FDA reviews say, well, that feature hasn't been cleared. That's happened. And usually these are for features. So features are usually in my mind the trigger point for you know, 510(k). You're not changing the intended use okay. But you may add a feature and somebody, your competitor will see it and they need to react. And then they'll like submit a 510(k). And then it'll be discovered that that particular feature was never cleared. So I think with the PCCP, like you said earlier, if you change the intended use, it's ineligible. Okay and if you wanted to add that feature in your intended use, then clearly you have to do a 510(k). But so those are two scenarios that I've seen where, you know, FDA has in that catch up 510(k). You know, was how do you get from version. You know, if they're looking at version eight. And the last time FDA reviewed it was version five, they might want to know how you got cleared for that. And then a competitor not turning you in. It's using your device as a predicate device. But if you have that PCCP, then you'll be covered for these types of things perhaps.
Yujan Shrestha: Sorry, I'm just kind of sketching out my thoughts for like the key takeaways, at least for me. So is this an accurate statement to you? So like before, before the PCCP, you know, say we have version one that's already been cleared. So this is 510(k) or De Novo four version one cleared. And then you know, we're going to we're going to take that and we're going to deploy version one out there the post-market. And then we're going to plan what's going to go on the next version. And then we're going to implement that. And then we're going to V&V it. And then there's a question is it a note to file or a 510(k)? I really like this question will be answered. Maybe earlier on. But we're at this point here. And then we say, okay, well if it's a 510(k), now we got to wait six months before we or we can deploy right. Or you know, perhaps we can deploy with like a research use only kind of a gray zone. But that has been done flag. That's an even riskier approach really. Or you can say note to file and say that we're going to deploy this immediately as a note to file. We're going to take the risk. Right. Risky road. But now we're saying there's another branch in this where when we do the version 2.0. planning and we know what features we're going to build, we could turn that around and submit a 510(k) for just a PCCP with the version 2.0 plan. And in a perfect situation, the PCCP will be authorized just in time for the implementation to be completed.
J. David Giese: Yeah.
Yujan Shrestha: And like in theory, we save six months without taking any risks.
J. David Giese: I think I think this is an interesting alternative to like doing it during the like the first 510(k) for V1. If you know, if you've planned it that far, you can do it then, right? But if you don't or like then yeah, like this could shift the timeline quite a bit.
Yujan Shrestha: So yeah.
J. David Giese: I guess the downside.
Yujan Shrestha: Yeah.
J. David Giese: So you have to have it planed out quite well in advance. Like you have to have all the requirements and verification protocols and everything all mapped out at a high level of detail that often with our clients, they don't know that well like it's not until they start building it that they're like really understand exactly how they want to do it. And that would be a problem either way.
Yujan Shrestha: Okay. Yeah, yeah. Because then now there's like all the PCCP, we can't modify the PCCP but is it close enough. Yeah we did. Is it. That's yet another question. What's making even more complicated. But no I get it. Like I totally get like it. It is another this is another option. Like what other ways can we use a PCCP like you'd mentioned? We know ahead of time. I think if the first one had it, the first 510(k) had a PCCP in it, then yeah, you could kind of like if you knew V 2.0 features when you submitted, you could add that or say so I would probably say 510(k) for the 1.0 and PCCP for, I don't know, V 2.0. Like this is yet another option right. So then we have well this will be this will be cleared here and it would lead to a deployment of this. And then it would kind of in a time timeline wise, it would lead to the immediate deployment of version 2.0. If you get it, if you can predict software developments, which I'm not good at doing that. That's why we have agile, because I don't think anyone's good at doing that. But, you know, if you can foresee that…
J. David Giese: I'm sorry to interrupt, but I think we there's a question that actually came in a little bit ago. So Cooper hey, by the way, thanks for your comment. I think I saw you commented on my post yesterday. Thanks for your question. I hope you're doing well. And so the question is, if you submit a PCCP as described in the patient monitoring example, once you reach the specified parameters after retraining the AI model, are you able to retrain it again in the future to improve performance using the same justification, I'm asking this to determine if including the plan to modify the model enables you to make multiple distinct changes to the model,
Future of PCCP in Industry and Strategic Planning 🔗
J. David Giese: Or only retrain it to bring it within the specified parameters the once so in. Certainly, I think there's some flexibility in how you write the PCCP, but you certainly can have multiple different modifications that you describe, not just one. And in fact, like in the one PCCP that I've done there is actually like I think 12 changes. So for a whole variety for multiple models and so, so certainly and I think probably there's a whole class of changes where you can say, hey, we're going to just keep improving the performance. Following this approach and I would imagine that the FDA would be fine with that. So I think the answer is yes, you are able to continue retraining it under that PCCP. So anyway, I hope hopefully that answers your question.
George Hattub: Here's another scenario I wanted to mention. You know, we're always talking about the FDA. And you know, what about if you're a small company okay. And you have some really innovative technology, you got clearance and then you're making changes, okay. And then the bigger company is interested in licensing your technology. So in my experience with the larger companies that the regulatory teams plan on in, those companies are tougher than the FDA. And they'll look at your files and question during the due diligence period. Why did you make this change, and what was your rationale for not doing this or that? And then I've also seen situations where the regulatory teams have a lot of influence in the decision. And it's not that they won't eventually license that technology, but they'll use it as a negotiating tool to get a lesser price offer. And the rationale will be is because we're going to have to submit catch up 510(k)s or we're going to be exposed. So I think that this is something to think about. And one of the reasons why you might want to use this tool, because we are going to be changing the nature of anything engineered is to make it better. And you know, like you said, having this already gone with the FDA is not only going to save you money, but it could save you a significant amount of money. Let's say if a larger company wants to utilize your technology, license it, you follow what I’m saying?
Yujan Shrestha: Yeah. I think like it will decrease the overall risk. Right. So then yeah, like to.
George Hattub: Treat it as an investment.
Yujan Shrestha: Oh yeah. Yeah.
George Hattub: Yeah I guess it would be one more why you'd want to do this.
J. David Giese: So I kind of maybe make sure I'm following it in the same way that you're having a nicely run QMS is important for the tool okay. But it also in addition to just doing it because you can get dinged in an audit, it also makes you a better acquisition target for a larger company because they're going to that.
George Hattub: Absolutely, and not only that, they might allow you to be more autonomous. If your quality system's a mess, they might say, we need to take over this company as opposed to running it as an autonomous unit. I've seen that.
J. David Giese: That makes sense. And so you're saying if you take the risky route and and do notes to file for everything, then the larger company would come in and say, hey guys, this is not appropriate. We need to do a big catch up 510(k). And not only that, yeah, we're going to pay you less money.
George Hattub: Yeah.
J. David Giese: You because we know there's a big mess. We got to clean up now.
George Hattub: Yeah it won't it won't kill the deal. And they don't mention that to you. It goes back to the business development person. The person is interested in your company. And so the regulatory people once again they have a lot of say so and they know all the tricks of the industry. And so, you know, these larger companies, I find them more risk averse than even the FDA because they're worried about the reputation of the company. They're worried about their position. And the way that can blow back on you is just like how you describe. So I always advise clients, think big, think three steps ahead of, you know, where you are today. And you know, once again, you know the companies they're require you the better your regulatory file the position over you QMS. It's only going to help you. It's like an investment. So maybe the PCCP is an investment. Like a Pep.
Yujan Shrestha: Yeah. No. Totally other. Yeah I'm trying to sketch out like think what we what kind of visually for same like ideally you just need one PCCP to cover a specific type of change. So you say low risk and low cost. So this is going to be low risk high cost. So the only option we had before right. So for like changes that like retraining the algorithm to improve performance. If you can write the pieces in a general way that same PCCP, the is going to be a gift that keeps on giving. And then you can reuse this consistently, right, for all future versions, for future changes. Like you know, if the if all your changes were just retraining and improving, then you could like use that same PCCP or at least just not have to worry about that pain again, that type of pain again. Versus before that PCCP, the only two options you had were two. A whole bunch of note to files, hope for the best rate was just kind of taking on a little bit more risk there, or you want to de-risk it completely, then do a 510(k) each time because you know, you have to pay the FDA user fee and all the preparation and whatnot. But now this other option, you could get that first PCCP, authorized and there's no expiration date to the authorization as long as you follow the modification procedures and everything in that.
George Hattub: Interesting.
Yujan Shrestha: And that'll be interesting. And this is slightly different than this like version 2.0 planning. In this case it is PCCP is really like a one time use PCCP.
J. David Giese: Yeah.
Yujan Shrestha: Like for like specific features. But you can have more general purpose changes like retraining or more. Yeah. More abstract changes.
J. David Giese: Yeah. I think for the ones where you could reuse it multiple times, it'll probably need to be more even, I would imagine even more defined and precise in what you could do. I think for the software like feature changes, where it's really like we're adding something to the software, not retraining, you certainly could do the same thing for like maybe four features, right? And then you'd have your all the requirements for all of them, the verification part. And then when you would submit a PCCP for that, you'd want to split out those modifications into separate groups so that FDA could review them separately. And they might say, hey, we accept these three, not this fourth one. So maybe that would be a situation where you could reuse it, but in a much more limited capacity than in retraining but testing.
Yujan Shrestha: Interesting, so you're saying like speculative PCCP or like because just because you have a PCCP doesn't force you to implement that change. You could you could decide to not implement the PCCP.
J. David Giese: Yeah, for sure. But the downside is you'd have to plan out all these features in a pretty high level of detail in order to submit it. And then if you're not really serious about implementing it, it might not be worth it doing that. Like you might instead wait for the approach that you have here with you do the you just submit the PCCP upfront before the implementation like that maybe would it make you just repeat that four times? Possibly. I don’t know, but.
Yujan Shrestha: Right, right. Yeah. Like in this case it's just saving you six months versus in this case it is saving you cost as well. Like I guess if you. Yeah you're right. There's like yet another branch somewhere here. It's like PCCP for version one and multiple not really speculative but I guess what would you call it. I'll just call it that for like another terms like speculative processing PCCP version 2.0 and beyond like yeah, we think we need these features. So we're going to go ahead and just included pay the user fee once and need the quantum computer to come up with all of the possibilities into just some everything. I don't know, but that could be an interesting application of this that we got to try out.
George Hattub: Yeah, push the limits.
Yujan Shrestha: I'm sure that was not an intended usage of the features. Do you want to be kind of using it off label? But it's worth a shot anyway, we are at time, so definitely thank you for everyone's participation. And I think that was a very interesting conversation. And yeah, love to see how we end up utilizing this new tool. I think it has the potential to save a lot of time and money. I think it's a net positive. Like I can see now how sure, there are probably of some categories. What were a note to file before? Now we're going to do a PCCP. Maybe it's a little bit more work there. But on the flip side there were a lot more 510(k)s that now can be PCCP.
Closing Remarks and Upcoming Discussions 🔗
Yujan Shrestha: So I think the net average result is less work for industry, lower cost, faster time to market. So it seems really positive to me.
J. David Giese: And I didn't mention this, but we have a big PCCP article. Well, it's not that big, but a recently big FAQ. And I'll probably take a passive updating it now that the new guidance is out. And I'll probably maybe I'll take some of these ideas here and kind of incorporate different questions to in that maybe another comment.
Yujan Shrestha: Right.
J. David Giese: Yujan, we should I was thinking the next two weeks were off rate for the holidays. So definitely next week. And I'm happy do that. So maybe let's start again on the that would be the ninth. And then our, our company retreat is the week after that. So I think we'll also skip the 16th. And I was going to propose that we plan out maybe the next couple months of topics and, and try to promote it a little more before maybe that would see like a, maybe a discussion for, for the recording. But yeah we should. Yeah.
Yujan Shrestha: So you'd like to get a PCCP authorized for upcoming cut podcast episodes.
J. David Giese: Exactly. So exactly.
Yujan Shrestha: Yeah. All right. Okay. Well, well, thanks a lot for buddy. Yeah. You guys have a great rest of your week and have a happy holiday season. We will likely see you guys again in the new year. So Merry Christmas everyone. Happy new year.
George Hattub: Bye now. Great.
Yujan Shrestha: Bye. All right.
