Russian developers created a platform for self-testing of AI medical services

August 07, 2020

Experts from the Center for Diagnostics and Telemedicine have developed a platform for self-testing services which is based on artificial intelligence and designed for medical tasks, such as for analyzing diagnostic images. The first working prototype of the platform is hosted on the popular GitHub service, and developers from all over the world can take part in its improvement by adding verification criteria depending on the purpose of the services. Sergey Morozov, CEO of the Center for Diagnostics and Telemedicine, spoke about this at the thematic week dedicated to artificial intelligence which was part of the program of the European Congress of Radiology (ECR 2020).

Before implementing a service based on artificial intelligence (AI) into routine clinical practice, it is necessary to test it for technical readiness, as well as to verify whether it meets the stated characteristics. It is called analytical validation of the algorithm. The services that have passed it are allowed to be integrated into medical systems, including city healthcare.

Integration is a complex and expensive process, so it becomes a barrier for many teams that cannot guarantee the required accuracy and speed of the algorithm processing data of the system into which they are integrated. Currently analytical validation is performed manually. Manual validation allows accidental or deliberate deviations from the approved test program, as well as manipulation of datasets, and also can potentially put different test participants in unequal conditions.

To solve these problems and automate the verification process, ensuring trust of users, specialists of the Center for Diagnostic and Telemedicine have developed a platform that allows developers of AI-based services to independently conduct preliminary tests (analytical validation) of their algorithms. A prototype of the platform has been hosted on the GitHub, and the first version of the service for exchanging datasets and data analysis results has already been uploaded.

The platform provides an opportunity for the unlimited number of accesses to single samples of data instances from the test set in order to fine-tune algorithms. It has uniform rules of use, and it is possible to test several services simultaneously. At the same time, the platform records the time that the software spends on data processing (time-study), and the developers receive an automatic report on the results of testing, - explains Sergey Morozov, CEO of the Center for Diagnostic and Telemedicine.

By automating the entire process on the self-testing platform, the human factor is minimized, which makes data manipulation (to improve results) impossible. In addition, the comparison of the service's verification results with the reference data is absolutely transparent - the developer can see what metrics were used, and how the final result reflected in the report was calculated.

Anyone can take part in improving the platform and add necessary metrics to it, which will be used to evaluate the algorithm's performance for certain medical purposes (for example, for analyzing radiographs or mammograms). However, the addition of the platform will be monitored - the only metrics that have scientific justification will be included in the platform operating on the basis of the Center, - notes Nikolai Pavlov, the developer of the platform, Head of Dataset Labeling Conveyor of the Medical Informatics, Radiomics and Radiogenomics Sector, Center for Diagnostics and Telemedicine.

The creators of the platform invite developers of AI algorithms, programmers and researchers to take part in updating and improving the platform in order to develop a uniform, universal, and user-friendly tool for self-testing of artificial intelligence algorithms intended for medical purposes in the international community. At the moment, there is no such tool aimed specifically at the clinical implementation of services based on AI technologies.

Center of Diagnostics and Telemedicine

Related Telemedicine Articles from Brightsurf:

Changes in outpatient care delivery, telemedicine during COVID-19 pandemic
To understand how telemedicine compensated for declining outpatient volume and geographic variation in changing patterns of outpatient care, researchers examined telemedicine and in-person outpatient visits in 2020 among a national sample of 16.7 million people with commercial or Medicare Advantage insurance.

Show rates for asthma visits during COVID-19 increased thanks to telemedicine
A new study being presented at this year's virtual ACAAI Annual Scientific Meeting reveals that ''show rates'' for children with asthma - how often parents brought their kids to an appointment rather than being a ''no show'' - increased with the use of telemedicine during four months of the pandemic.

From 84 days to 5 hours: Telemedicine reduces dermatology consult time
Allowing primary care doctors to take photos and send them to dermatologists improved access to specialty care.

Health care use drops during pandemic; switch to telemedicine creates disparities
One of the first studies to quantify the cuts in elective medical care experienced in March and April found that the number of mammograms and colonoscopies dropped by more than 65% during the period.

Utilizing telemedicine in the ER can reduce wait times and patient length of stay
Telemedicine has become more common given the current global pandemic.

Telemedicine saves chronic pain patients time and money
Patients who saw a pain medicine specialist via telemedicine saved time and money and were highly satisfied with their experience, even before the COVID-19 pandemic, according to a study being presented at the ANESTHESIOLOGY® 2020 annual meeting.

Interventions stem antibiotic prescribing rates in telemedicine
Two different interventions both worked to significantly reduce the rate of inappropriate antibiotic prescriptions made by physicians in a telemedicine practice, a new study led by Children's National Hospital researchers shows.

Telemedicine may well outlast the pandemic, say mental health care staff
The Covid-19 pandemic has brought about rapid innovation in mental health care, and the move to telemedicine is likely here to stay to at least some degree, but new research led by UCL and King's College London cautions that serious barriers still need to be overcome.

Assessing telemedicine unreadiness among older adults during COVID-19 pandemic
This study uses 2018 data from the National Health and Aging Trends Study to assess how common it is for older adults in the United States to be unprepared to access video or telephone telemedicine because of disability or inexperience with technology.

Telemedicine can help safety net providers expand access to medical specialists
The use of telemedicine has grown rapidly during the coronavirus pandemic, but it's unclear whether those gains will ge permanent.

Read More: Telemedicine News and Telemedicine Current Events is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to