Related Posts
ALC Medical Marijuana Oversight Committee
December 14, 2022
Sen Sample: We’re going to call this ALC Medical Marijuana Oversight Committee together. Representative Fite, do you have anything?
Rep L Fite: I do not. Thank you.
Sen Sample: Okay. We’re going to get right to it because we need to be out of here before 10:00. And so Ms. Doralee, you’re up. And you’re going to review the rules of ABC.
Chandler (ABC): Doralee Chandler, Director of the Alcoholic Beverage Control. We have some rules to present. These are rules that we were asked to issue and create for the Medical Marijuana Commission by the Supreme Court, and when one of the Supreme Court decisions came down in the Carpenter Farms matter. And so these rules set out the guidelines for rulemaking as well as hearings and motions to be heard in front of the commission. It is a mirror of– what we tried to do is mirror the Administrative Procedures Act to put it in the rules so that the entities and licensee applicants would be aware of the policies and the procedures and the steps to go forward so they would not be going back and forth between the APA statute and the medical marijuana rules.
Sen Sample: Representative Ferguson, do you have a question for Ms. Chandler?
Rep K Ferguson: Oh, I’m sorry. I hit my button accidentally.
Sen Sample: Okay. Are there anyone that wishes to question Director Chandler? Seeing none, I need a motion to review the rules. Have a motion and a second. Representative Ferguson. Thank you. All right, Ms. Chandler.
Rep K Ferguson: I do have one quick question. I don’t know if you can answer it or not, Director, but regarding the software issue, I’ve had some entities talk to me about in terms of the tracking, how many fines have– if you can answer this question, has ABC maybe issued some of the entities, the dispensaries– can you tell whether it’s because of the software or it’s not or could you talk about that a little bit?
Chandler (ABC): So I can’t give you a definitive answer at this point on that without reviewing additional documentation. What we do is when we get a report of a violation from enforcement or a law enforcement agency– if it’s some type of theft violation, it may come from a law enforcement agency– but generally, our reports come from the ABC enforcement division. And when they came over, based upon the documentation that we received, we issue what we call an offer of settlement. The offer of settlement goes to the entity with the option to pay the fine that is based upon the violations listed in the offer, or they can request a hearing. If they request a hearing, that provides them with an opportunity, after I set that hearing, to come in and provide evidence as to whether or not their defense, essentially their defense, to the allegations that have been set out against them. If they have come in and established with my confidence that it is a software issue, then we have been looking at that and trying to dismiss, issue warnings, or handle it in a different manner or reduce the fine based on anything that we can establish is confidently based upon a software issue. So we have started looking at this evidence that they present to allow them an opportunity to demonstrate to us that it is a software and not a training issue of staff or an inability to use their software correctly.
Rep K Ferguson: Can I have a follow-up, Mr. Chair? Can you tell me, do you have any idea of how many incidences we’ve had that was regarding software issues or an entity claim that it was a software issue? I guess that’s a better question.
Chandler (ABC): I would say that at least 75% of our offers that go out, we get some indication or response or statement that they believe it is a software glitch or issue, depending on the violation. If it is an inventory violation, oftentimes, their response is that it is a software issue. And that’s when we generally have the hearings and walk through as to whether it’s just an employee that has made an error versus an actual software glitch where there may be some type of duplication or reduction based upon a glitch in the system.
Rep K Ferguson: But we don’t have any numbers as to how many times this happened?
Chandler (ABC): I have not kept track of the numbers on those issues. It’s something I could go back and look at our violations and ascertain, but I don’t have the document.
Rep K Ferguson: Is it possible at some point in time you guys could take a look at that?
Chandler (ABC): Absolutely.
Rep K Ferguson: Okay. Thank you. Thank you, Mr. Chair.
Sen Sample: Senator Chesterfield.
Sen Chesterfield: Thank you, Mr. Chair. And good morning. How are you?
Chandler (ABC): Good Morning.
Sen Chesterfield: Good. Doralee, will you be available at the end of this meeting in case there are questions that arise for you as a result of the testimony going forward today?
Chandler (ABC): Absolutely.
Sen Chesterfield: All right. Thank you, Mr. Chair. Oh, and one quick question. We know about the kerfuffle, if you will, as far as one of the sites is concerned. One’s going to lose theirs, and the other is going to get theirs. What role are you guys playing to bring that to some kind of reasonable conclusion?
Chandler (ABC): Are you referring to the lawsuit that’s currently in place?
Sen Chesterfield: Yes, ma’am.
Chandler (ABC): It is currently under a stay pursuant to– Supreme Court has issued a stay. And so at this point in time, we’re not taking any action on that until the Supreme Court finalizes their ruling and hears the matter.
Sen Chesterfield: Thank you. Thank you, Mr. Chair.
Sen Sample: Okay. Seeing no other questions, you’re dismissed. Now item D, we have discussions regarding BioTrack software concerns. Bill Paschall, you’re first up. Mr. Paschall, I know you’re not for people, and so if you would introduce yourself and who you represent and then pass the microphone to your left.
Paschall: Good morning. I’m Bill Paschall. I’m here on behalf of the Arkansas Cannabis Industry Association. I’m here to kind of just set the tone and let these three gentlemen who are in the field, two dispensary folks and one cultivator, talk to you about their experiences with the software, the ARStems program that BioTrack runs for the state. To my left is Robbin Rahman. He is with Harvest Cannabis in Conway, and then Kyle Maiten, who is with Natural State Medicinals down in White Hall. They’re a cultivator. And then Adam Maldonado with Delta Cannabis from West Memphis. My remarks are going to be brief. They’re basically what I’ve said to you before. ACIA is agnostic when it comes to who provides software for seed-to-sale tracking in Arkansas. I have the privilege of talking to colleagues around the country, frequently from other states, that sell marijuana. And I can tell you from those conversations, there is no perfect software system out there. There is no perfect provider.
Paschall: With that said, I do continue to get complaints occasionally from our members about customer service, about communication, and sometimes about training. So I think there’s still room for improvement with the existing company that provides the software. They are working to improve. They’ve offered to do some training with our members in the coming weeks. Today at 10 o’clock will be our last training session with ABC that we’ve done around the state. We’ve had four. This will be the fourth. And we’re working with their folks to better understand the ARStem system, what they’re looking for, how to enter information into the system so that we have fewer violations. I would say to your question, Representative Ferguson, about 70% of the violations I have seen, and I’ve looked at those through May. I think it’s the last report I got from Doralee. They’re generally software-related violations. We have odor violations and other things at times, but the vast majority of them are dealing with some kind of coding error, software error. So I will finish with that and hand the mic over to Mr. Rahman and let him talk to you about his experience with his dispensary in Conway.
Rahman: Good morning, and thank you for hearing us today. Again, Robbin Rahman, I’m the executive director of Harvest Cannabis in Conway, Arkansas. It’s a family business. My parents own it. And we have a pretty extensive experience with BioTrack, both the ARStem system and BioTrack Core, which is the product that is used at the point of sale by some of the dispensaries in the state. We’ve used it since 2019, so about three years. And I’d say our learning curve has been fairly steep but steady. We’ve experienced quite a few issues related to ARStems and/or BioTrack, and they fall into kind of three different categories. I’d say the first one are kind of user error, and that’s our own inexperience with the product, our own sort of just the price of business. You make mistakes, just incorrectly using the software. So we’ve experienced a lot of that. Some of it is our hardware. We had a server crash in 2020, which essentially deleted, I would say, 100% of our customer data for six months. And that was a huge setback. And we learned a lot of things as a result of those user errors. The second category are truly issues with the software that require the input of BioTrack’s helpdesk. You call a phone number, you reach a person on the other line who can sort of help you sort through these software issues. Now, for those issues, you sort of have to start with the assumption that no one in any business likes a help desk. You don’t like calling them. You do not like talking to them. So if you start with that, and then you also add to that these issues usually manifest on a very busy day, we’re either short-staffed, or we’re fully staffed, but everyone is fully occupied. So you have to sort of step away from that busy day and figure out this very critical software issue. And sometimes helpdesk folks are not moving with the same sense of urgency that maybe you need them to move.
Rahman: However, all things being equal, I can’t really complain too much about the helpdesk. They’ve been available and responsive and have helped us resolve the issues almost all the time. And then some of the problems are just sort of unknown. No one really knows what the issue is. Whether it’s ARStems, which is the statewide system, or BioTrack core, which we use, sometimes they just don’t communicate. And you might be down for– when they don’t communicate, you cannot proceed with any transaction. You literally have to stop what you’re doing to figure that out. And sometimes, it happens probably every two to three weeks, and it might be for as little as five minutes. It might be for as long as a full hour or more than that. And there’s really no great answer for why that occurs. In fact, just this past weekend, we had an outage. I think it lasted for several hours, and it was a failure to communicate between ARStems and BioTrack core, and it resulted in 12 different anomalies. And an anomaly is essentially a product is recognized on one system but not– a product sale is recognized in one system but is not recognized in the other. And so you have to go in after the fact and kind of do some cleanup, and that’s an adjustment, and then that usually triggers some sort of investigation by the state as to why we made those adjustments. So those are the three categories.
Sen Sample: Yes. The Senator wishes to know what ARStem– that’s the state system.
Rahman: Of course. Correct. So when you step back– the entire system is built on the concept of tracking every single gram or seed of product from seed all the way to sale. And so there is an infrastructure that sits on top of the entire industry, and it’s called ARStems. It’s what the state contracts with BioTrack for. That system, ARStems, is the way the state can look and see where every single gram of product is at any point in its life cycle up to the point that is sold out into the public. That is distinct from the point-of-sale system that each of the operators use or the cultivation inventory system that the cultivators use. Those are contracts between the operator, whether it’s a dispensary or a cultivator, and BioTrack. So those are very distinct things. But those two systems have to communicate. What I put into my inventory at Harvest has to match and be identical to what the state sees in ARStems. And so there’s constant communication back and forth. It’s real time. And if at any point that real-time link is severed for any reason, then you cannot complete a transaction. Or if you do complete a transaction, there’s going to be discrepancies between the two that then require either further investigation, some type of adjustment that you make manually into your system. In order to do that, you have to be very, very on top of your audit process. You have to constantly be aware of where you are in that sale, whether they were communicating. And the system is pretty good at telling you when they’re not communicating. And so you can kind of go pins down until you figure that out, but it’s not always the case. Does that explain kind of where everything sits? So to go back, those are the three categories of our issues that we’ve experienced.
Rahman: But on the whole, as Mr. Paschall just mentioned, I think no system is perfect. We have our complaints, but I don’t think that replacing it or seeking out a new provider would really improve the situation. I think we have three years of institutional knowledge, and that’s very valuable to us. And so switching to some other platform, I think, would be a step back, and we’d be hard pressed to improve our situation in a short time. Notwithstanding our experience, I’ve heard all the same complaints that you’ve heard, that Bill has heard, and so I acknowledge that not everyone is happy. And so if that is the case, then it seems like we should endeavor, at very least, to try to improve the relationship. And if you’re going to improve the relationship, I think there are a couple of things that really need to be understood on the front end before we go about that process. And the number one thing that I want to leave you with from our perspective is, no matter what happens, when the state contracts with BioTrack for the ARstem system, when we contract with BioTrack for our individual point-of-sale system, if there’s ever anything that’s wrong, the only party that suffers are the operators. We are the ones that will either get fined because we have not been able to figure out what the situation was. If we can’t complete a sale because there is an outage, because the systems aren’t communicating, we are the ones that suffer from lost sales, having to shut down, and things like that. So if there’s one thing that I could share with you as you think about this relationship and how to improve it, it’s just always keep in mind that we are truly the end user, the ones that will suffer. So I just hope you can keep that in the back of your mind as you’re moving forward. If you have any other questions, I’d be happy to answer them. But otherwise, I’d yield the floor.
Sen Sample: I have one question. You mentioned that your system had crashed.
Rahman: Yes, sir.
Sen Sample: Okay. And you have backup, probably off-site backup, right?
Rahman: At the time, we did not, and we learned a lot from that experience. It really crippled us for several days. We thought that the hard drive that we were using was sufficient for what we expected to be the pace of business, and we were wrong by a factor of 10. And very quickly, that hard drive just was not adequate. We have since upgraded to a much more commercial-strength hard drive that has a sort of managed IT services contract that sits on top of it to constantly check for vulnerabilities, to do off-site warehousing of our data, and things like that and mirror some of the hard drives so that should we experience another outage, we won’t have to go through the same process again. Again that was a tough lesson to learn, but it happens.
Sen Sample: I can just imagine how tough that lesson was. Representative Ferguson.
Rep K Ferguson: Thank you, Mr. Chair. Mr. Rahman?
Rahman: Yes, sir.
Rep K Ferguson: Okay. I want to follow up on something that Chairman said. When your system crashed, but it was your system, it was on you guys and not with the state or– the state and your system was not talking to one another, it is on you guys and not–
Rahman: That’s correct.
Rep K Ferguson: Okay. And the other question, have Harvest Cannabis, have you guys ever been fined?
Rahman: Yes, sir.
Rep K Ferguson: Was it your fault?
Rahman: I guess that depends on who you ask. There have certainly been instances in the past where there’s really no question that the rule says what it said. And we varied from the rule, whether it was because of intentional conduct or unintentional conduct or just not paying attention or just the price of business. We’re by no means the leader in the state in terms of volume, but we do 14,000 transactions every month, and we have to track every single product down to the gram. And if at any point in those 14,000 transactions you were off by even a gram, that can generate a fine. And so you have to have extremely good business practices to make sure you don’t have any fines. And I cannot say that we have always had the practices we have now. I think we’ve gotten to a good place, but we have not always been there. And it has taken three years to figure out what the right behavior is. And that’s pretty much for every aspect of our business. We’re truly entrepreneurs, truly kind of developing the business as we go.
Rep K Ferguson: So it was the fault of your company.
Rahman: In some instances, I will acknowledge that it was our fault. I think in other instances, our position remains that the interpretation of the rule was too intense.
Rep K Ferguson: Didn’t have anything to do with software?
Rahman: That is a very difficult question to answer. To put the blame of an issue entirely upon the software would not be fair to the software. It would not be fair to the user.
Rep K Ferguson: Let me ask you this way. Did you guys appeal?
Rahman: We have never appealed the fine, no sir.
Rahman: All right, thank you.
Paschall: Representative Ferguson. If I can follow up from the industry perspective, most of our folks have been hesitant to appeal because it takes a lot of time and energy. They have to hire lawyers. Some of them have started appealing. The appeals process, I think, has helped improve communication between ABC and our folks. Hence, we’ve had these trainings this year in the last three months, and I appreciate ABC for doing that. I think it’s going to make a big difference going forward with how our licensees understand the system, what is expected of them and get them trained. There are instances, there’s no doubt, because I’ve talked to all of them, where it was a software issue. And Doralee has been relatively kind, I can’t say in every case, but she’s been fair for the most part when those are brought to her attention and when the appeals are made. But there are instances where folks have been fined for software issues.
Sen Sample: Senator Chesterfield.
Sen Chesterfield: Thank you, Mr. Chair. I guess I’m concerned when I say you guys are the ones that suffer. It’s the patients that suffer. When you cannot get the product to the individuals who need it the most. It is indeed the patients that are suffering. And I keep hearing, well, BioTrack is doing this or it’s not doing this. And you get here today and it’s like a big kumbaya. So wherein lies the issue? If it’s not BioTrack, it’s not you, per se, it’s not the state, wherein lies the issue that is precluding the individuals who really matter? And to me, those are the patients. Wherein lies the problem?
Rahman: I think that’s a fair comment that the patients also suffer if the outage prevents us from completing transactions. That’s not always the case. Sometimes the transaction itself is not accurately reported, and so that might generate a fine. The patient would never know that it had happened. However, we would be the ones that would pay the fine. So I think we’re both on the same page here about where the true harm lies. And again, I don’t want to give you the impression that this is a kumbaya. I’m trying to give you a realistic impression of what our experience has been. I acknowledge that our experience has not been the same as every other operator and that others perhaps have had a tougher go at it, whether it’s because where they are, maybe– I have no explanation for it. As far as where the truth lies, unfortunately, this is such a complex, highly regulated area that the idea that there is a single point of failure that you can point to and say, this is the problem. I don’t think that you’re going to find that here. I think, again, 14,000 transactions, every single product tracked down to the gram. There are so many points of failure along that process, from seed to sale. The idea that it is one thing I think isn’t realistic. I think there are certainly cases where the software has failed or you just can’t explain what the issue is, and perhaps they’re doing the best they can. I don’t know. I know that we do the best we can. And when there are unexplained software failures, I don’t really know what the answer to that is.
Sen Sample: Okay, seeing no more questions for Mr. Rahman, if you would, give your name for the record.
Maiten: My name is Kyle Maiten. I’m with Natural State Medicinals.
Sen Sample: Pull the microphone closer to you so–
Maiten: Sorry, Chairman.
Sen Sample: –so everyone can hear.
Maiten: Is that better?
Sen Sample: That’s good.
Maiten: Thank you for this time this morning. I don’t have a lot to add to what Mr. Rahman said. However, I will give you an example of some of the subpar customer service that we have had in the past. It would be generous to say at times 1 out of 20 calls will get answered by an individual that will actually be able to set up an appointment. A lot of times, at least in the past, it has been that you would call and it would go straight to the voicemail and you’d leave a voicemail and hopefully they’d get back to you. At times it would be a day, other times it’d be a couple of weeks. I have also experienced instances where I would have a tech working on a problem that they were supposed to be fixing. They have a team viewer in which allows them to get onto our system and go through the problem, apply the fix, and during the course of that, the tech would drop off and I wouldn’t be able to get a hold of them. They would simply, for lack of a better word, ghost me. This has been quite frustrating, as you can imagine. If we have an appointment set up and we take time out of our day to allow them to work on our system, it’s frustrating when that fix is not applied as quickly as possible. The software has its lack of functions, but that is going to be an issue with any software. There’s no perfect software out there, as Mr. Rahman said, but we have many reports that we have in our software that we are really unable to utilize. I don’t know if this is something that is still something that BioTrack is working on, or if it is just something that is something that’s available in other states that we do not have available, but that is something that we would like to see improved.
Maiten: There’s a lack of data retrieval in certain inventory forensics reports. If you try and go back further than about a year, oftentimes it will crash the software because you’re trying to pull information that the software cannot access for whatever reason. We have seen improvements in the software. However, we have also been told that we have certain updates coming that have not come to fruition yet. I’ve just actually sent an email out to an individual with BioTrack yesterday asking for a follow up on that. We had something that was supposed to be implemented in September. We have not seen that come into play as of yet. Another issue is, while there is training that BioTrack says that they offer, I think it should have been offered on the front end. We basically had to train ourselves up on how to use this and how to figure out all the conversions and all the different movements in the software. They have videos on their site. However, these videos are outdated. They really need to bring them up to the most current version, and we would hope that they would be more in depth. That’s really just a glance across the surface of what the software can do and what it should do. And on top of that, they have a manual on the website. However, we feel that is outdated and should be updated. Really, those are the only issues I have at the moment that I can share with you. But I’m happy to answer any questions that you might have.
Sen Sample: Representative Speaks?
Rep Speaks: Thank you, Mr. Chair.
Sen Sample: If you would, pull your microphone a little closer, I can hardly hear.
Rep Speaks: Okay, thanks. You keep talking about they. Is this the state that you’re talking about? The IT works for the state or–?
Maiten: That would be BioTrack that I’m referring to, the company that implements the software.
Rep Speaks: So they are the ones who done the software. Okay, thank you.
Maiten: Yes.
Sen Sample: Representative Ferguson, you have a quick question?
Rep K Ferguson: Yes sir, I have a quick question. You mentioned something about a technician not showing up. That technician was with BioTrack, not with the state?
Maiten: That’s correct, with the BioTrack.
Rep K Ferguson: Were you down in terms of your facility for any period of time?
Maiten: We were not down as far as the software goes. We had a problem with a particular item in the software that needed correction, but it didn’t affect day to day functionality of other issues.
Rep K Ferguson: Okay, thank you.
Sen Sample: Thank you. Seeing no other questions. Mr. Maldonado, if you would, give your name. State your name for the record.
Maldonado: Good morning. My name is Adam Maldonado. I am the assistant manager at Delta Cannabis Company in West Memphis, Arkansas. I’m here today to provide you with information about a catastrophic event caused by a power surge in and around West Memphis, Arkansas. I’ll also provide the subcommittee with an overview of the problems and challenges we’ve experienced as a result of the Seed To Sale tracking program, including problems at the point of sale, incorrect inventory tracking, known as ghost sales, and poor and unreliable customer service. On August 26, I was working in our store and I observed the lights began to flicker, and that continued until we lost power. Approximately 45 minutes after the power failure, the server that was connected to BioTrack began to shut down completely. Prior to this incident, we were under the assumption or the impression that BioTrack backed up our inventory daily and in the event we would lose power, we could use that backup to help get us back online.
Maldonado: This was not true. In fact, after the power was restored to the store we could not connect to BioTrack and therefore could not make any sales and had to send sick patients home without their medication. At around 1:30 that day, we discovered that BioTrack had not backed up any inventory data for over a year. We made several attempts to get back online and retrieve the data, only to confirm that BioTrack data was either not backed up or all that was in the hard drive was corrupt and unusable. We attempted to resolve our concerns with Biotrack’s assistance, but later in the afternoon – this was a Friday – BioTrack customer service completely ghosted us. We submitted several requests for help to BioTrack, and we failed to get a response from them again until Sunday. Because our files were corrupted and we could not get any help on how to proceed forward, we had to remain closed for the entirety of the weekend, and then we had to rebuild all of our inventory from scratch, including all of our patient data as well, causing us to lose approximately around $100,000 in revenue.
Maldonado: What’s harder for us to figure out or quantify is what we may have lost in revenue from future sales of people who chose to go elsewhere for that weekend and then stayed with that dispensary for the future. As assistant store manager, I’ve also noticed daily inventory problems with BioTrack software. I’ve repeatedly personally experienced BioTrack software incorrectly tracking patient allotments, random and unexplained changes in inventory weight, and disappearance and reappearance of inventory. For example, I believe Mr. Rahman mentioned failure to connect to ARStems during the middle of a patient transaction. And what happens on the back end is it will create a retail customer sale that deducts from a patient’s allotment, and it deducts from the inventory, but then it creates a phantom sale and item, which then creates more problems. So the most recent example of one of these issues occurred on December 2 when there was a disconnect from ARStems, which caused us, as well as other dispensaries around the state, as far as I’m aware, to go offline for upwards of two hours. Caused us to have to deploy additional manpower to fix all of the inventory issues that were caused from BioTrack. I would be happy to provide any additional details or answer any questions if you need. Thank you.
Sen Sample: Senator Rice, you have a question?
Sen Rice: Thank you. You stated that BioTrack didn’t back up their data for a year. Where were you able to get that information confirmed from?
Maldonado: Could you repeat the question, please? I didn’t hear the last part.
Sen Rice: Who supplied you that information that you can verify that?
Maldonado: From my understanding, I believe that it was from BioTrack.
Sen Rice: BioTrack admitted they hadn’t backed up for a year?
Maldonado: That the hard drive had not backed up. Yes, correct.
Sen Rice: Thank you. Appreciate that.
Sen Sample: Representative Crawford.
Rep Crawford: Thank you, Mr. Chair. My question is, who requires you to use BioTrack?
Maldonado: The state. The state, I believe.
Paschall: BioTrack runs the ARStems program for the state. Each of the dispensaries can choose whatever POS system they prefer. If they don’t–
Rep Crawford: What is POS? I’m sorry.
Paschall: Point of sale. I’m sorry. Their point of sale system. When you go to the register and ring out that interfaces with ARStems. That is up to the individual dispensary. The vast majority of our folks, I think, use BioTrack’s system, their POS system. But there are others that don’t. And we often hear complaints about non-BioTrack POS systems having difficulty talking to ARStems.
Rep Crawford: Okay, follow up please.
Sen Sample: Quickly.
Rep Crawford: What other, other than BioTrack, what other software is available through the state?
Paschall: BioTrack is the contractor for the state to run the ARStems system, the big system. It is up to the individual dispensary to choose what point of sale system they use, which has to interface with that.
Rep Crawford: Okay. Thank you.
Sen Sample: Senator Chesterfield.
Sen Chesterfield: Thank you, Mr. Chair. I’m hearing more concern from the Good Day Farm in Pine Bluff and the Delta Cannabis in West Memphis. Is any of that because of a lack of good IT in your area, or is it simply that you’re not being responded to as you should be?
Paschall: Senator Chesterfield, there is differing levels of sophistication by company as to the training and the resources they have actually put into it. So we do see some folks that probably make more mistakes than others. They likely have not invested as much in training as others. And then Good Day, as you know, they’re kind of the leader in the field in terms of sales in Arkansas now from a cultivator standpoint. So they run more volume through. So that may be why you’re hearing from them. Their volume is much greater than some of the others. But there’s varying levels of sophistication when it comes in training, when it comes to each licensee.
Sen Chesterfield: And the ability to upload the data is predicated on the amount of IT infrastructure in the area, is that correct? We don’t have the gigabyte capability in some places that we do in others. That’s where I’m trying to get to, because if there are poor people in Pine Bluff who really need it and can’t get it. West Memphis, it’s more difficult to get, I’m understanding. But Conway is fine or more fine than the individuals on this end of the table. Would that be correct?
Rahman: It’s tough to say. I think the information that is being passed from our point-of-sale system to the ARStems system is not heavy in terms of the actual bytes that are being transmitted across the Internet. However, it is completely true that access to high-speed internet in an uninterrupted manner is very important because, again, ARStems, which is the state seed to sale tracking system, must communicate down to the second with our point-of-sale system so that they are a mirror image of one another. The sale is completed at our store and inventory is removed from our records. There must be a message that is transmitted to the state in real time that removes it from the state system, because it’s not just the state’s inventory that must be tracked. It’s also that individual patient’s allotment that must be adjusted so that that patient cannot purchase two and a half ounces from my store and they go next door to his store and purchase another two and a half ounces. It all has to sort of tie out in real time. So access to high-speed internet is very important. And uninterrupted high-speed Internet is very important. Whether the information is a lot of information, I don’t think that’s the case, but.
Sen Chesterfield: Last question on this issue. BioTrack has a contract. I think it’s a seven-year contract. Is that correct? And when does that contract expire?
Paschall: Memory serves me, 2023.
Sen Chesterfield: I’m seeing no, in the back.
Paschall: ’24. November 2024.
Sen Chesterfield: So they still have time to get it together so that we don’t continue hear about BioTrack. Are we going to continue? I guess we’ll ask BioTrack whether or not we’re going to continue, to hear from them. But this contract lasts through 2024. December?
Paschall: November, she said.
Sen Chesterfield: November 2024. Thank you, Mr. Chair.
Paschall: And Senator Chesterfield, I’ll continue to have communication with BioTrack and ask them to continue to step up their training from their end, and they’ve indicated in willingness to do that.
Sen Sample: Members, I want to remind everyone that we have to be out of here by 10:00. And so please keep your questions to a minimum. And you’re up next, Representative Wardlaw.
Rep Wardlaw: Thank you, Mr. Chair, is it out of order if we brought the Health Department and ABC to the table for my question? I have a question about the contract and who mans the contract? And I think the state would answer my question better than the individuals.
Sen Sample: Can we get this bunch out of the way?
Rep Wardlaw: Yes, sir.
Sen Sample: And then I’ll bring them in real quickly before we go to BioTrack.
Rep Wardlaw: Thank you.
Sen Sample: That’s fine. Representative Ferguson.
Rep K Ferguson: I’ll be brief, Mr. Chairman. Thank you. Mr. Paschall, I want to get back. The entities don’t have to use BioTrack, correct?
Paschall: Not for the point of sale. They do for the ARStems. Everything has to go into the state system to track.
Rep K Ferguson: So they can use another software system, but that software system has to be able to talk to the state?
Paschall: Correct.
Rep K Ferguson: Which is operated by BioTrack.
Paschall: Correct.
Rep K Ferguson: Okay, thank you.
Sen Sample: Thank you. Seeing no other questions, you’re dismissed.
Paschall: Thank you, sir.
Sen Sample: Doralee, I think? Is someone from the Health Department here? Here we go. Doralee, I got a real quick question while everybody’s getting the table. Who is the state’s broadband provider? Is that ARE-ON or do you use a different network?
Chandler (ABC): Sorry, I couldn’t hear you.
Sen Sample: Who is the ABC’s internet or broadband provider? Is that ARE-ON? Or do you have another broadband system?
Chandler (ABC): So for our IT works, we use the Office of Information Management Services over at DFA, and then we also work in conjunction with DIS.
Sen Sample: Okay. All right. If you would, state your names for the record.
Thompson (ADH): Charles Thompson, attorney, Arkansas Department of Health.
Shue (ADH):I’m Laura Shue. I’m general counsel for the Department of Health.
Sen Sample: Thank you.
Chandler (ABC): Doralee Chandler, director of ABC.
Sen Sample: Representative Wardlaw.
Rep Wardlaw: Thank you, Mr. Chair. So who and where does the BioTrack reside? Does it reside at Health Department or does it reside at ABC?
Chandler (ABC): Currently, the contract resides with the Health Department. We have communicated with them that when the contract is up in 2024, ABC will have the funding available now that the medical marijuana industry is operational. We will take over bidding out and going forward with the contract after it expires.
Rep Wardlaw: So we remember Lieutenant Governor, now Attorney General Griffin’s silo effect over DHS. And it seems like that’s where we’re at here is we have a silo over at the Health Department, a silo over ABC, and not the communication that we need. How hard would it be to move all of the oversight to ABC or all of the oversight to the Health Department so that all of this is in the same silo?
Thompson (ADH): Thank you, Representative Wardlaw. I think ABC and ADH, we do communicate. We have weekly calls that we review issues that for both of us on both of our regulatory authority under Amendment 98. Amendment 98 says that ADH will do patient registry cards, labeling standards, testing standards. And so when it comes to actual BioTrack usage, we use the patient portal. That’s what generally we use. The seed-to-sale stuff when it comes to the tracking, the seed-to-sale, that’s ABC. Because of way we’re split, I don’t want to say siloed, but we have very specific authority under Amendment 98. I don’t know that you could take all of that regulatory capability and put it into one. To your question, we do communicate with ABC very regularly. I think you’re still limited by what Amendment 98 says, who shall do what. And that’s what the people have told us has delineated and split that out.
Rep Wardlaw: And it makes sense to me that you guys do the cards, you guys look at that side. It just seems that the authority of who’s over the contract and who can man fixing some of the issues with the contract has been kind of up in the air. And I was asking around yesterday, and one said, “No, it’s the other.” The other one said, “No, it’s the other one.” And that is a silo effect, by the way. And that’s very concerning to me that we were getting those answers while we were asking. So it does make me feel a little better that ABC is going to pull that contract in 2024. I think that’s probably a good fix for what we were finding looking through this issue.
Thompson (ADH): Representative, we agree totally. And that’s when ABC stepped up to say they would take that, we agreed with them that would probably be the best when it comes to the contract and seed-to-sale system.
Rep Wardlaw: Thank you. Thank you, Mr. Chair.
Sen Sample: Okay, seeing no other questions, you are dismissed. Moving to item E, reporting regarding BioTrack software. We would ask you to come up at this time. Please identify yourself for the record, and you may proceed after that.
Afaneh: Good morning. Thank you. My name is Moe Afaneh. I’m the VP of BioTrack. I would like to start by thanking this committee for giving me the opportunity to address this body here today. I want to go ahead and begin by addressing what Mr. Maldonado was talking about. There was a document circulated, and this was complaint number one on that document, specifically with the incident where they had the server crash and the corruption. We just wanted to clarify that the corruption and the backups that he was referring to, the system automatically backs up the local commercial system separate from ARStems. They were using an unapproved way of backing it up off to a different drive. When they had the power surge, the data was corrupted. Now what Mr. Maldonado stated was correct, that on Friday these activities happened. My team was not as responsive as they should have been on Friday night. And by Sunday morning, the situation was escalated to me. That’s when I started working directly with Keith, who is the IT provider for Delta’s organization. I started working with him directly. He asked me to review the corrupted drive as well. We took it, and we escalated internally to different engineers to try to see if there’s a way to recover some of the data. But there was corruption. That data that he is referring to is separate from the data within ARStems. We did an emergency migration and we spun them up inside of our hosted Amazon servers to help get them back up online as quickly as possible. Once we do that, we can initiate what’s called a synchronization between their local system and ARStems that will now pull the information that has already been reported into ARStems into their local system. Now, what that sync does not bring back in is local information that they would have stored within their system themselves. Notes about patients, their label set up, receipt set up, how their products were priced. Now setting up pricing on the products is not the same thing as saying that the inventory had to be rebuilt. You literally click synchronization and it pulls down everything that the state has visibility into at that given moment. All the inventory, the rooms, the sales activity from inception, all that is captured and recorded.
Afaneh: During this situation, I continued to work with Keith, the gentleman who is the IT provider for Delta through Sunday, Monday, and Tuesday. As I was working with him, we identified that the staff on site was trying to what they thought was like helping the situation. They started making adjustments to inventory that created a knock-on effect for additional things. Instead of trying to address the problem, they were trying to do a quick little workaround which created some additional effects. All that is to say that by the time I identified that and made the recommendation to Keith that says, “Hey, they should stop this behavior to alleviate these problems,” we were back on the road to recovery. And by the time we wrapped up Monday, everything was back in working order the way it was expected to be. And by Tuesday morning, they were operating back as normal. Again, I take responsibility. We are far from being perfect. There’s always a potential for issues to occur, for problems to happen. We’re constantly looking to improve what we’re doing, how we’re doing, how we’re communicating with the market, providing additional training, looking for opportunities to improve best practices within these facilities. So we are always looking for that type of feedback. I know that we are short on time, so if you bear with me, I’m just going to try to move through some of these items that were listed in this document. We provided in this document– I’m not sure if the entire committee has seen this, but we provided a graph of our uptime. Now this shows you that we are nearly near 100%. We’re in the 99% plus range. Some of those gaps in time are due to pre-approved maintenance windows through ABC for updates, maintenance on the system, additional activity that needs to happen. There are some times where these things are outside of our control. The ARStems is hosted on an Arkansas.gov website. Meaning if something happens to that Arkansas.gov website, it becomes inaccessible. Even though the ARstem system in the background is still there and running, they just can’t reach it because .gov website is inaccessible for something outside of our control. That doesn’t mean that we don’t have random outages or situations that occur. Anytime those occur, we look for feedback from the permittees themselves to let us know if they are having an issue. Specifically going back to the Delta team.
Afaneh: Keith again, anytime there’s any sort of situation, he instantly sends me a text message. I’m very responsive to him. We’ll get on the phone. At times we’ve had him fail over to a secondary internet provider that has resolved the issue at times. I know that they use Ritter as well as another internet provider. We saw that happen recently, a week ago or so, where they switched to the different provider and that seemed to resolve their issue. So there is maybe some infrastructure challenges at play with who the internet providers they are using from time to time. There’s a statement here about miscalculation, about patient limits. There is a report that ABC has access to where you can plug in a patient ID number. It will show you every single sale, where it will happen, what the product was, how much is deducted from the patient limit. If there’s any questions about that, we’re more than happy to do an analysis to look into if there was some sort of miscalculation. But we have not seen that happen or anytime we’ve investigated, we haven’t seen that scenario. There’s a number of statements in here about changing weights, inventory amounts changing unexpectedly. We’re more than happy to investigate these if they have a specific example or a facility that can point us in the right direction and we can do some analysis. But these are kind of vague for us to kind of try to speak to. There’s a statement in here about BioTrack inventory rounds down fractional weight. This is a little bit out of context. Another point of sale provider was pushing data incorrectly into ARStems. So for example, if a permittee was making an adjustment for three and a half grams, this other point-of-sale system was reporting it as 4 grams. So a half a gram times, 10, 15, 20 times, it begins to add up. This is a scenario of where they pushed bad information in and that was kind of the information that was recorded in Stems, which created a discrepancy between what they physically had versus what was reported to Stems. There’s a tool within the commercial BioTrack system. So not the ARStems, but they have a tool in there for synchronization. So if a sale is seen captured inside of ARStems, but not in their local system, they can use the sales synchronization. And what this does is the local system communicates to ARStems and says, “Hey, show me the differences.” And if there is a sale in Stems, it’ll try to pull it back in to automatically keep both systems matching in real time. I spoke with Robbin yesterday. We are going to do some follow up with his managers to make sure they understand how to use that tool. So any future situations like that, they can help themselves and quickly address the situation. There’s another statement in here about us not being accessible on the weekends or after hours. Our standard support hours are essentially 10-7, Monday through Sunday, essentially Eastern Standard Time. Now, above and beyond that, that’s for scheduled support. We also have support that is available till 1 a.m. Eastern, Monday through Friday to address any escalations, any after hours activities that are needed to happen as well as before the 10 a.m. clock, we actually have support that begins to come online at 8 a.m. to start proactively looking at any requests that have come in the previous day or things that need to be quickly addressed with customers.
Afaneh: We also publish on our website a number of phone numbers and email contact information for licensees to be able to reach out to us. It is also on the actual login screen of ARStems with an email and a phone number if they didn’t see it on our website. There’s a statement in here about the manifest indicating the state of Washington. We generated an example. It clearly says the state of Arkansas. This was something very, very old that probably goes back a number of years. Inability to transfer certain types of inventory. Again, that’s kind of a vague statement. There’s a certain workflow of what inventory can be transferred based off of the type of permittee they are and the type of action they’re trying to do. That is with the approval of ABC in terms of how that workflow was created. So more than happy to work with this individual permittee, if there’s a specific example we need to look into, and I know we’re down to time. I just want to thank this committee again for giving me the opportunity to speak in front of you. More than happy to answer any questions.
Sen Sample: Senator Rice, you’re recognized.
Sen Rice: Yes, and I’m going to have to slip over to another committee, but can you briefly give me the answer on the state system has some type of interruption. Do you have a way to document when that happens? The reason I’m asking, if the state’s at fault and things do happen, I don’t want people fined– I have a very distasteful thing when state doesn’t do their job and we don’t sometimes. And we expect citizenry to live under one set of rules and we excuse ourselves sometimes. The other thing is, can you substantiate what the gentleman said is that BioTrack did not back up a secondary backup for like a year.
Afaneh: So in regards to the backup sir, the system generated the backups. Their backups were corrupted due to the power surge. So the files actually exist on there from their perspective. From a non technical lens, you see the file exist on the drive as you’re looking at it on the screen, but when you try to unpack that backup, if you will, to restore the data, there was corruption in various different tables. We attempted ourselves to try to figure out if we could slice out part of the corruption so we could at least put back some of it, but that was simply just not possible.
Sen Rice: Okay, the other part of my question.
Afaneh: I’m sorry.
Sen Rice: The other part. Can you document when it’s the state is at fault of their system either going down or something that’s not linking? Can you substantiate any way to show us who’s at fault on these interruptions?
Afaneh: Yes, we can take a look at that and review our logs, sir. We have alerts that are constantly tracking the system. If it’s up, if it’s available, if the API, which is the part of the software that the different point of sale systems use to actually communicate back and forth with ARStems, we can take that as an action item and get you that information, sir.
Sen Rice: That’s all the time I’m going to take time. I’ll let others have time. Thank you.
Sen Sample: Senator Chesterfield, you’re recognized.
Sen Chesterfield: December 2 seems to be a very important day as far as difficulty is concerned. Could you speak to what happened on December 2?
Afaneh: On December 2, I believe that there was a situation, and again it was the gentleman, Keith from Delta. He shot me a message right away. We quickly jumped on the system to see what was going on. It seemed like that there was some sort of high load affecting the system. We quickly took action and resolved the situation. I can go back and check my communication and let you know exactly how much that window was, but it was fairly short in time.
Sen Chesterfield: All right. Thank you.
Afaneh: You’re welcome.
Sen Chesterfield: Thank you, Mr. Chair.
Sen Sample: Senator Hickey, you’re recognized.
Sen Hickey: Yes, sir, I’m over here. Thank you. Just one question. One of the gentleman and you’ve addressed some of it with the weights and the grams, but he mentioned something called phantom transaction, ghost transactions. Have you been made aware of that before today or before this report?
Afaneh: So I was not aware that he had a ghost transaction. It can happen. Essentially what occurs is they submit a transaction to the state. The system really cares about the patient ID and less so about the name. So it was submitted up to ARStems, recorded, and then in the downstream to recording their local point of sale system. There could have been some sort of interruption in that communication. It records the sale, but it puts retail customer as a holding place to identify that something has occurred with the sale that needs further investigation.
Sen Hickey: Okay, so whenever his reference is a ghost transaction or phantom transaction or whatever that may be, that’s not necessarily just saying that there was a transaction that took place out there and there wasn’t an individual involved. Because from my standpoint, as simple as it may be, things like that always lead me to think about, well, do we have a security issue or this or that? And if I understand you correctly with what you just said, what that could possibly be would be somebody that was there who may not have been approved that they came in to the store. Is that what you’re saying? If not, you’re going to have break that down a little more simple for me.
Afaneh: Sure. So when a sale is submitted, the system cares about the patient ID. So we know who to deduct against from their limit. That sale went up to ARStems and in the downstream back into their local system, there could have been some interruption. It still retains that patient ID is the one the sale is attributed to. It just didn’t put their name on it because it’s really tracking against the patient ID itself. So it’s not that the sale just created itself out of nowhere, it’s related to a transaction, but something happened to that transaction that needs adjustment.
Sen Hickey: That’s helpful. Thank you, sir.
Afaneh: You’re welcome.
Sen Sample: Okay, seeing no other questions, you are dismissed.
Afaneh: Thank you, sir.
Sen Sample: We appreciate everyone who came today and we are now adjourned.