ORIGINAL STUDY

Survey-based Evaluation of the Use of Picture Archiving and

Communication Systems in an Eye Hospital—
Ophthalmologists’ Perspective
Downloaded from http://journals.lww.com/apjoo by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1A

Thomas Chi Ho Lam, MRCSEd(Ophth)
y, Jerry Ka Hing Lok, MRCSEd(Ophth), FCOphthHKy,

Timothy Pak Ho Lin, MBChBy, Hunter Kwok Lai Yuen, FRCOphth, FRCSEd
y, and
WnYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8K2+Ya6H515kE= on 03/04/2024

Mandy Oi Man Wong, FCOphthHK, FRCSEd(Ophth)
y

example, comparison with previous results. Subspecialty played an

Purpose: Picture archiving and communication system (PACS) is a
important role in evaluating PACS.
medical imaging system for sharing, storage, retrieval, and access of
medical images stored. Our study aimed to identify ophthalmologists’
Key Words: electronic ophthalmic imaging, ophthalmology diagnostic
views on PACS, with the comparison between 3 platforms, namely
tests/investigations, optical coherence tomography, picture archiving and
electronic patient record (ePR), HEYEX (Heidelberg Engineering,
communication system, visual field
Switzerland), and FORUM (Zeiss, US), following their implementation
in an eye hospital for common ophthalmic investigations [visual field, (Asia Pac J Ophthalmol (Phila) 2022;11:258–266)
optical coherence tomography (OCT) of retinal nerve fiber layer and
macula, and fluorescein/indocyanine green angiography (FA/ICG)].
Methods: An online survey was distributed among ophthalmologists in a P icture archiving and communication system (PACS) is a
medical imaging system that provides storage and access of
images from multiple modalities. It consists of image acquisition
single center. Primary outcome included comparison of PACS with
paper-based system. Secondary outcomes included pattern of use and devices, storage archiving units, display stations, computer pro-
comparison of different PACS platforms. cessors, and database management systems, which are integrated
Results: Survey response rate was 28/37 (75.7%). Images were most by a communications network system.1 Since the introduction of
commonly accessed through ePR (median: 80% of time, interquartile the concept by Lemke et al in 1979,2 PACS has been widely
range: 50 to 90%). adopted in the field of radiology worldwide, in the hope of
All systems scored highly in information display items (median scores achieving “filmless radiology”.3–6
7.5 out of 10) and in reducing patient identification error in investiga- In recent decades, the rapid evolution of ocular imaging
tion filing and retrieval during consultation compared to paper (score technology brought about increasing reliance on graphical infor-
7.0). However, ePR was inferior to paper in “facilitating comparison mation to guide the diagnosis and treatment of eye diseases.7,8 In
with previous results” in all investigation types (scores 3.0 to 4.5). ePR Hong Kong, with an annual attendance of over 1 million in the
scored significantly higher in all system quality items than HEYEX public ophthalmology service, around 1.7 million ophthalmology
(P < 0.001) and FORUM (P < 0.022), except login response time images were handled every year.9,10 Before 2018, common
(P ¼ 0.081). HEYEX scored significantly higher among vitreoretinalu- ophthalmic investigations, eg, visual field (VF), optical coherence
veitis members (VRU) for information quality items for OCT macula and tomography (OCT) of retinal nerve fiber layer (RNFL) and
FA/ICG [VRU: 10.0 (8.0 to 10.0), non-VRU: 8.0 (6.75 to 9.25), macula, fluorescein angiography (FA), and indocyanine green
P ¼ 0.042]. angiography (ICG), were printed and filed into paper medical
Conclusions: Overall feedback for PACS among ophthalmologists was records manually. An electronic imaging system is imminently
positive, with limitations of inefficiency in use of information, for needed to reduce filing errors, facilitate retrieval, reduce storage
space, and to prepare for the advent of artificial intelligence.11 In
the 2000s, a large-scale web-based electronic patient record (ePR)
system with image distribution based on enterprise PACS was
Submitted July 27, 2021; accepted November 14, 2021.
From the
Hong Kong Eye Hospital, Hong Kong SAR, China; and yDepartment of developed in all public hospital services in Hong Kong.12 Since
Ophthalmology and Visual Sciences, The Chinese University of Hong Kong, 2018, Hong Kong Eye Hospital (HKEH), which had near 250,000
Hong Kong SAR, China
The study results were submitted for presentation in Hospital Authority Kowloon ophthalmic outpatient attendances per year, was chosen as 1 of the
Central Cluster Convention 2021. 2 pilot hospitals to incorporate common ophthalmic investiga-
The authors have no conflicts of interest to declare.
Address correspondence and reprint requests to: Mandy Oi Man Wong, Hong Kong tions into 3 types of image archiving and display systems, namely
Eye Hospital, 147K, Argyle Street, Kowloon, Hong Kong. E-mail: mandyo- ePR and 2 commercially available PACS platforms. The investi-
[email protected], Hunter K.L. Yuen, Hong Kong Eye Hospital, 147K,
Argyle Street, Kowloon, Hong Kong. E-mail: [email protected] gation results were uploaded onto PACS and viewed through
Copyright © 2022 Asia-Pacific Academy of Ophthalmology. Published by Wolters designated workstations in other clinical areas. According to the
Kluwer Health, Inc. on behalf of the Asia-Pacific Academy of Ophthalmology.
This is an open access article distributed under the terms of the Creative plan, PACS will be implemented in the remaining 9 ophthalmic
Commons Attribution-Non Commercial-No Derivatives License 4.0 (CCBY- centers in Hong Kong from 2021 onwards.
NC-ND), where it is permissible to download and share the work provided it is
properly cited. The work cannot be changed in any way or used commercially Similar to other electronic health information systems, PACS
without permission from the journal. is often installed at the enterprise level, involving significant
ISSN: 2162-0989
DOI: 10.1097/APO.0000000000000467 investment and change in workflow.13,14 Timely evaluation of

258 | https://journals.lww.com/apjoo © 2021 Asia-Pacific Academy of Ophthalmology.

 Asia-Pacific Journal of Ophthalmology  Volume 11, Number 3, May/June 2022 Use of Picture Archiving and Communication Systems

PACS at an early stage, with end-user perspective in particular, Subjects

would be beneficial for informing potential users of obstacles in We included all HKEH ophthalmologists or ophthalmology
implementation. Currently, there is a lack of evidence on clini- resident trainees, who had utilized patients’ ophthalmic imaging
cians5 view on the use of PACS in ophthalmology. Our study for clinical care from December 2018 to September 2020 and had
aimed to identify ophthalmologists’ view on the use of PACS, undergone the transition from paper-based to electronic ophthal-
with a comparison of different image management systems mic imaging system. Staff members who had not experienced
Downloaded from http://journals.lww.com/apjoo by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1A

following their recent implementation in our hospital. paper-based systems were excluded.

Survey Design/Instrumentation
METHODS The survey is designed with the framework developed from
WnYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8K2+Ya6H515kE= on 03/04/2024

This is a cross-sectional survey among ophthalmology special- DeLone and McLean model of information systems (IS) success,
ists and trainees to evaluate the use and satisfaction of PACS in which identified 7 domains for assessing the success of IS,
HKEH. The study was approved by the Research Ethics Committee including systems quality, information quality, service quality,
of Hospital Authority, Hong Kong Special Administrative Region, intention to use/use, user satisfaction, and net benefits.16 While
and followed the tenets of the Declaration of Helsinki. this model is widely applied in studying the success of informa-
tion systems or electronic medical record systems,17 we identified
PACS Setup in Hong Kong Eye Hospital no study that focused on ophthalmic imaging. The survey was
PACS in HKEH includes 3 systems: ePR, Heidelberg Eye therefore modified to fit into the context of our study. Intention to
Explore (HEYEX, Heidelberg Engineering, Switzerland), and use was not assessed as the use of system was compulsory.
FORUM (Zeiss, US). ePR is a pre-existing system implemented in Participants were asked to choose from a 10-point Likert scale
all workstations in HKEH since 2000 s, alongside all other public for items regarding ePR, HEYEX, and FORUM respectively in 4
hospitals and clinics in Hong Kong. Since December 2018, investi- domains, namely system quality and overall satisfaction, infor-
gation reports of OCT RNFL, OCT macula, VF, FA, and ICG mation quality, service quality/service support, and the use of
performed at HKEH can be accessed in ePR in Portable Document PACS. In the “use” session, participants were asked to indicate the
Format (PDF), instead of Digital Imaging and Communications in frequency of use for the individual functions of each system. An
Medicine (DICOM) format, which is a commonly used standard for “others” session was added to allow patients to choose from a list
communication in biomedical digital imaging systems.15 HEYEX, a of net benefits and issues commonly encountered, and to allow for
commercial eye care data management solution, is installed in all open-ended answers. After pilot testing among 5 ophthalmolo-
outpatient consultation rooms and selected inpatient areas in HKEH, gists of different subspecialties and years of experience for clarity
with a total of 29 viewing stations for investigations including OCT and usability, the link for the online questionnaire (Supplementary
RNFL, OCT macula, VF, FA, and ICG in DICOM format. Aside from Digital Content 1, http://links.lww.com/APJO/A120) was for-
displaying investigation reports in the same manner as ePR, compar- mally sent out to all relevant staff through staff email list. The
ison between reports, eg, VF/OCT RNFL reports can be achieved survey was self-administered and took approximately 45 minutes
using “layer” function. Interpretation of OCT macula (Heidelberg to complete. Anonymity was maintained without the need for
Engineering, Switzerland) is facilitated by various built-in functions, signature or identifiable personal information.
eg, volume scan, star scan, thickness map progression, and various
overlay tools. Similarly, interpretation of FA/ICG is also facilitated by Study Outcome
functions including operator-generated reports, individual series, The primary outcome of this study was strengths and lim-
magnification, and image inversion, etc. Another commercial plat- itations of PACS compared to paper-based system, in terms of
form, FORUM (Zeiss, US), has 5 viewing licenses for OCT RNFL system quality, information quality, and service quality/support.
and VF in DICOM format. It is mainly designated to enhance the Secondary outcomes included pattern of use of PACS and com-
management of glaucoma by providing Guided Progression Analysis parison of different electronic systems.
(GPA) of both VF and OCT RNFL (Zeiss, US).
Image quality of the investigations in ePR is comparable to Data Analysis
the paper printout, while that in HEYEX and FORUM would be Statistical analysis was performed with IBM SPSS Statistics
the same as that in the respective original imaging modality. v27 (SPSS Inc, Chicago, IL). Continuous variables were presented
Currently, all images uploaded to ePR can be read by all public in median and interquartile range (IQR). Paired and independent
hospitals in Hong Kong, which are managed by the statutory body continuous variables were compared with Wilcoxon signed-rank
of the Hospital Authority (HA). Likewise, those uploaded to the test and Mann-Whitney U test, respectively. Correlation between
ePR by other public hospitals under HA can be read at our continuous variables was studied with Spearman correlation. Sen-
hospital. On the contrary, for HEYEX and FORUM, images sitivity analysis was performed for scores in system quality and
uploaded at a particular HA hospital can only be viewed within information quality to identify the effects of the subspecialty of the
that hospital, but not by any other hospitals within HA. ePR can be respondent. P value < 0.05 was considered statistically significant.
reached by corporate-provided personal home devices with spe-
cial registration, while PACS cannot be reached by all home
devices. In compliance with corporate policy, images will be RESULTS
deleted from the system if the patient did not attend our service for
more than 6 years. The training was provided by information Demographics
system personnel in form of lectures and onsite support during the Twenty-eight out of 37 (75.7%) ophthalmologists or oph-
early stages of implementation. thalmology resident trainees responded to the survey. The median

© 2021 Asia-Pacific Academy of Ophthalmology. https://journals.lww.com/apjoo | 259

 Lam et al. Asia-Pacific Journal of Ophthalmology  Volume 11, Number 3, May/June 2022

age group was 31 to 40 years old (14/28, 50.0%), with 23/28 Sensitivity Analysis
(82.1%) having less than or equal to 15 years of experience in Subspecialty members working with posterior segment (vit-
ophthalmology. Further demographic data were listed in Table 1. reo-retina and uveitis, VRU) used HEYEX significantly more
than non-VRU members [% of time using HEYEX: VRU: 90.0%
Use of System (60.0 to 90.0%) vs non-VRU: 15.0% (10.0 to 30.0%),
Images were accessed through ePR for a median of 80% (IQR P < 0.0001], but not the other 2 systems (P > 0.475).
Downloaded from http://journals.lww.com/apjoo by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1A

50 to 90%) of time, while HEYEX was used for 20% (10 to 50%) of There was no difference in use between the 3 systems by
time. Fourteen doctors (14/28,50.0%) had used FORUM, yet the glaucoma/neuro-ophthalmology (GNO) members (P > 0.106).
majority (71.4%) of them used it as route of access for <10.0% of the
time. 82.1% (23/28) and 42.9% (12/28) used the electronic imaging System Quality
WnYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8K2+Ya6H515kE= on 03/04/2024

system for >30 and >50 cases per week respectively. The use of All 3 systems received positive feedback on system quality,
various functions of ePR, HEYEX, and FORUM for different with ePR having a significantly higher score in all categories than
investigations were listed in Table 2. Among them, PDF reports HEYEX, and in all except login response time when compared
were the most commonly used, for a median of 100%, 100%, and with FORUM. In comparison to paper system, all 3 electronic
90% of time for VF, OCT RNFL, and OCT macula respectively. systems scored highly (median of 7.0 to 9.0) in reducing patient
Display for star scan (70.0% of time) and volume scan (75.0%) for identification error in filing the investigations and in image
OCT macula in HEYEX were also commonly used. retrieval during consultation, with ePR performing better than
HEYEX and FORUM (Table 3A).

TABLE 1. Demographics of Respondents Sensitivity Analysis

Age, y 30: 9/28 (32.1%) For HEYEX, VRU members gave higher scores for satisfac-
31 to 40: 14/28 (50.0%) tion for login response time [VRU: 8.0 (6.8 to 9.0), non-VRU: 6.0
41 to 50: 3/28 (10.7%) (4.0 to 7.0), P ¼ 0.020], and reduction in patient identification
51 to 60: 1/28 (3.6%) error in filing the investigation during consultation [VRU: 9.0 (8.0
60: 1/28 (3.6%)
Ophthalmologic experience, 0 to 5: 11/28 (39.3%) to 10.0), non-VRU: 8.0 (6.5 to 8.0), P ¼ 0.017] (Table 4A).
y For FORUM, score for response time to log in was signifi-
6 to 10: 6/28 (21.4%) cantly lower among GNO members compared to non-GNO
11 to 15: 6/28 (21.4%) members [GNO: 5.0 (4.0 to 6.5), non-GNO: 7.5 (6.0 to 8.3),
16 to 20: 1/28 (3.6%) P ¼ 0.040] (Table 4B).
21 to 25: 2/28 (7.1%)
26 to 30: 1/28 (3.6%)
>30: 1/28 (3.6%) Information Quality
Position Resident (basic trainee): 2/28 (7.1%) All 3 systems showed satisfactory feedback in most of the items
Resident (higher trainee): 10/28 (35.7%)
in display of information, including correct chronological order,
Associate consultant: 12/28 (42.9%)
Consultant or
above: 4/28 (14.3%) display of all important information and of sufficient quality for
Subspecialty Vitreo-retina : 6/28 (21.4%) interpretation of data in all types of investigations (median scores
Pediatrics and squint: 6/28 (21.4%) 7.5 to 9.0 for VF, OCT RNFL, OCT macula, and FFA/ICG for ePR
Cornea/external eye disease: 3/28 and HEYEX, VF and OCT RNFL for FORUM) (Table 3B).
(10.7%)
When compared with paper system for the efficiency of use of
Glaucoma: 5/28 (17.9%)
Oculoplastics: 3/28 (10.7%) information, the results were more diversified. ePR was inferior to


Neuro-ophthalmology : 1/28 (3.6%) paper in facilitating comparison with previous results in all 4


Uveitis : 2/28 (7.1%) investigations (median scores 3.0 to 4.5). It also failed to make
General ophthalmology: 5/28 (17.9%)

interpretation easier than paper system in VF and OCT RNFL
Two members worked in both uveitis
(median score 4.0 and 5.0, respectively). Scores for making retrieval
and vitreo-retina team, with one of
them working also in neuro- of investigation more efficient than paper were moderate across all 4
ophthalmology. types of investigations (median scores 6.5 to 7.0). For HEYEX,
% time at HKEH per week 40.0 to 60.0%: 3/28 (10.7%) except failing to facilitate VF comparison better than the paper
70.0%: 1/28 (3.6%) system, it scored moderately (median score 6.0 to 7.5) in comparison
80.0%: 7/28 (25.0%)
to paper system in being more efficient in retrieving investigations
90.0%: 16/28 (57.1%)
100.0%: 1/28 (3.6%) and making the interpretation easier for VF, OCT RNFL, and FFA/
No. y on paper system 0 to 5: 12/28 (42.9%) ICG, and in facilitating comparison with previous results for OCT
6 to 10: 7/28 (25.0% RNFL and FFA/ICG. For OCT macula, HEYEX scored highly for
11 to 15: 5/28 (17.9%) all 3 categories comparing with paper system, with median score of
16: 4/28 (14.3%)
8.5 to 9.0, which were significantly higher than ePR. HEYEX also
No. y on electronic imaging 0 to 5: 26/28 (92.9%)
system scored higher mark than ePR in facilitating comparison with
6 to 10: 1/28 (3.6%) previous results for FFA/ICG. For FORUM, scores were modest
11 to 15: 1/28 (3.6%) (median scores 5.0 to 7.0) when compared to paper system.
Use of FORUM in reading 14/28 (50.0%) Although FORUM scored signficantly higher than HEYEX in
VF/OCT RNFL
facilitating comparison with previous results when compared to
HKEH indicates Hong Kong Eye Hospital; OCT RNFL, optical coherence paper system, the median score of 5.0 reflected the worse perfor-
tomography of retinal nerve fibre layer; VF, visual field. mance of both FORUM and HEYEX when compared to paper.

260 | https://journals.lww.com/apjoo © 2021 Asia-Pacific Academy of Ophthalmology.

 Asia-Pacific Journal of Ophthalmology  Volume 11, Number 3, May/June 2022 Use of Picture Archiving and Communication Systems

TABLE 2. Use of System

No. staff aware of the function Frequency of use (median, IQR)
1. VF
ePR/HEYEX PDF generated by operator 27/28 (96.4%) 100.0% (80.0 to 100.0%)
HEYEX comparison of multiple test mode 22/28 (78.6%) 10.0% (0.0 to 32.5%)
FORUM overview and browser 13/14 (92.9%) 0.0% (0.0 to 25.0%)
Downloaded from http://journals.lww.com/apjoo by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1A

FORUM GPA 13/14 (92.9%) 0.0% (0.0 to 10.0%)

2. OCT RNFL
ePR/HEYEX report generated by operator 28/28 (100.0%) 100.0% (90.0 to 100.0%)
HEYEX multiple display using “layer” function 21/28 (75.0%) 0.0% (0.0 to 40.0%)
WnYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8K2+Ya6H515kE= on 03/04/2024

FORUM structure-function GPA 13/14 (92.9%) 0.0% (0.0 to 5.0%)

3. OCT macula
ePR/HEYEX report generated by operator 28/28 (100.0%) 90.0% (50.0 to 100.0%)
HEYEX overlay tools (eg, calipers, area measurement, etc) 25/28 (89.3%) 30.0% (0.0 to 75.0%)
HEYEX magnification tool 24/28 (85.7%) 15.0% (0.0 to 50.0%)
HEYEX volume scan 27/28 (96.4%) 70.0% (0.0% to 100.0%)
HEYEX star scan 28/28 (100.0%) 75.0% (0.0 to 100.0%)
HEYEX thickness map progression 26/28 (92.9%) 30.0% (0.0 to 92.5%)
HEYEX thickness profile 28/28 (100.0%) 15.0% (0.0 to 50.0%)
HEYEX 3D view 26/28 (92.9%) 0.0% (0.0 to 27.5%)
4. FA/ICG
ePR/HEYEX PDF generated by operator 28/28 (100.0%) 100.0% (57.5 to 100.0%)
Individual series on HEYEX 27/28 (96.4%) 20.0% (0.0 to 80.0%)
HEYEX overlay tools (eg, calipers, measure circle, etc) 26/28 (92.9%) 0.0% (0.0 to 27.5%)
HEYEX magnification tool 26/28 (92.9%) 10.0% (0.0 to 20.0%)
HEYEX image inversion 25/28 (89.3%) 0.0% (0.0 to 55.0%)
HEYEX image modification (eg, contrast, brightness, etc) 26/28 (92.9%) 0.0% (0.0 to 15.0%)
5. Other functions 1/28 (3.6%)
Using “previous” arrow to compare with previous OCT macula scan n/a
ePR indicates electronic patient record; FA, fluorescein angiography; GPA, guided progression analysis; ICG, indocyanine green angiography; IQR, interquartile
range; OCT, optical coherence tomography; PDF, portable document format; RNFL, retinal nerve fiber layer; VF, visual field.

Sensitivity Analysis users reporting poor resolution of investigations (17.9%) than

For HEYEX, VRU members gave significantly higher scores HEYEX (P ¼ 0.025).
than non-VRU members for displaying investigations in right
chronological order [VRU: 10.0 (8.0 to 10.0), non-VRU: 8.0 (6.75
to 9.25), P ¼ 0.042], as well as for most items in information DISCUSSION
quality session for both OCT macula and FFA/ICG (Table 4A). Our study evaluated the ophthalmologists’ view on 3 recently
There was no significant difference in scores between GNO implemented image archiving systems in an eye hospital, dem-
members and non-GNO members for all items in VF and OCT onstrating overall positive feedback and identifying areas of
RNFL in FORUM (P  0.096) (Table 4B). limitation. To our knowledge, there had been no published report
on PACS from ophthalmologists’ perspective.
Service Quality/Support Compared to paper system, PACS gained most recognition
Twenty-three out of 28 (82.1%) respondents received 0 to on the improvement of data archiving, reflected by high scores in
2 hours of training. 92.6% (25/28) needed technical support 0 to reducing patient identification error in filing and retrieving inves-
2 times per month in the first 14 days of use. Issues encountered tigations, displaying in right chronological order, and reducing
when seeking support were listed in Table 5A. Overall, suffi- loss of investigation records. This was echoed by reported out-
ciency of training before implementation and technical support come on the implementation of a DICOM-compatible workflow
were moderate [score 7.0 (6.0 to 8.0), 7.0 (5.0 to 7.8), respec- in an ophthalmology clinic, which showed 50% reduction in
tively]. Confidence in using the system was good for ePR and need to enter or edit patient information into the testing device,
HEYEX, scoring 9.5 (8.0 to 10.0) and7.0 (7.0 to 8.0)respec- and reduced need to manage misfiled images by 76% in a survey
tively, but suboptimal for FORUM [score 5.0 (2.5 to 7.3)] among technicians.14 Limitations identified in our study involv-
(Table 5A). ing technical aspects, such as unsatisfactory login time, were
similar to those reported in radiology, for example, low speed of
Other Benefits and Issues of PACS network.18 Same as physicians,5 training was a challenge in our
Other benefits and issues of PACS were listed in Table 5B. hospital, with nearly half of the respondents reporting a lack of
The most agreed benefits included enhancing communication time to seek technical support. On the other hand, certain lim-
with colleagues (15/28, 53.6%) and quality of clinical decisions itations of PACS identified in our study were not shared among
(13/28, 46.4%) compared to paper system. For the issues encoun- radiologists or physicians. While the latter reported improved
tered, unacceptable login time were noted in both ePR (25.0%) productivity and reduced reporting time with PACS,6 in our study,
and HEYEX (50.0%), while 28.6% of ePR users noted diffculty in except among VRU subspecialty members, there was no signifi-
identifying location of investigations, and 39.3% of HEYEX users cant advantage of PACS over paper system on the overall
noted unacceptable loading time. ePR had significantly more perceived efficiency of data usage, for example, in being more

© 2021 Asia-Pacific Academy of Ophthalmology. https://journals.lww.com/apjoo | 261

 Lam et al. Asia-Pacific Journal of Ophthalmology  Volume 11, Number 3, May/June 2022

TABLE 3. System Quality, Overall Satisfaction, and Information Quality

Median score
ePR (n ¼ 28) HEYEX P value FORUM P value P value
(n ¼ 28) (HEYEX (n ¼ 8) (FORUM (HEYEX
compared compared compared
Items with ePR) with ePR) with FORUM)
Downloaded from http://journals.lww.com/apjoo by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1A

TABLE 3A. System Quality and Overall Satisfaction

About the system:
1. Easy to learn 9.0 (8.0 to 10.0) 6.0 (6.0 to 8.0) <0.001


6.0 (5.0 to 7.0) 0.010
0.403
<0.001


0.003

0.032

WnYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8K2+Ya6H515kE= on 03/04/2024

2. Workstations easily available 9.0 (8.0 to 10.0) 7.5 (6.0 to 8.0) 5.0 (5.0 to 7.0)
3. Satisfactory response time to log in 8.0 (8.0 to 9.0) 6.0 (4.0 to 8.0) <0.001


6.0 (5.0 to 8.0) 0.081 0.204
4. Stable and seldom crash 8.0 (7.0 to 8.8) 7.0 (5.0 to 8.0) 0.001

6.0 (5.0 to 7.0) 0.022
0.230
5. Reduce patient identification 8.0 (8.0 to 9.0) 8.0 (7.0 to 9.0) 0.025
7.0 (7.0 to 8.0) 0.014
0.102
error in filing the investigations
(compared to paper system)
6. Reduce patient identification error in 9.0 (7.0 to 9.0) 8.0 (7.0 to 8.0) 0.039
8.0 (7.0 to 8.0) 0.031
0.317
file retrieval during consultation
(compared to paper system)
TABLE 3B. Information Quality
About information from the system:
1. Display investigations in right chronological order 8.0 (8.0 to 9.0) 8.0 (7.0 to 10.0) 0.841 8.0 (6.0 to 8.0) 0.719 0.892
2. Reduce loss of investigation records 9.0 (8.0 to 9.8) 8.0 (7.3 to 9.8) 0.072 8.0 (7.0 to 9.0) 0.121 0.516
(compared to paper system)
3. Display all important information one looks for from the reports/images
VF 8.0 (7.0 to 9.0) 8.0 (7.0 to 9.0) 0.467 8.0 (7.0 to 9.0) 0.396 0.796
OCT RNFL 8.0 (7.0 to 9.8) 8.0 (7.0 to 9.8) 0.299 8.0 (5.5 to 9.5) 0.131 0.131
OCT macula 8.0 (7.0 to 9.0) 9.0 (8.0 to 10.0) 0.041

FFA/ICG 7.5 (7.0 to 8.0) 8.0 (7.0 to 10.0) 0.016

4. The display is of sufficient quality for interpretation
VF 8.0 (7.0 to 9.8) 8.0 (7.0 to 9.0) 0.732 8.0 (8.0 to 9.0) 0.795 0.434
OCT RNFL 8.5 (8.0 to 9.8) 8.0 (6.3 to 9.8) 0.032
8.0 (6.0 to 9.0) 0.169 0.932
OCT macula 8.0 (6.0 to 9.0) 9.0 (8.0 to 10.0) 0.009


FFA/ICG 7.5 (5.3 to 8.8) 8.0 (6.0 to 10.0) 0.284
5. More efficient in retrieving investigation reports/images (compared to paper system)
VF 6.5 (4.3 to 7.8) 6.5 (2.0 to 8.0) 0.986 6.0 (3.0 to 7.0) 0.798 1.000
OCT RNFL 7.0 (6.0 to 8.8) 7.5 (4.3 to 8.0) 0.344 7.0 (5.0 to 9.0) 0.866 0.865
OCT macula 6.5 (3.0 to 9.0) 9.0 (6.0 to 9.8) 0.027

FFA/ICG 7.0 (3.3 to 8.0) 7.0 (3.0 to 9.0) 0.878
6. Make the interpretation easier (compared to paper system)
VF 4.0 (3.0 to 7.0) 6.0 (2.0 to 8.0) 0.098 6.0 (3.0 to 8.0) 0.199 0.205
OCT RNFL 5.0 (4.0 to 8.0) 6.0 (3.3 to 8.0) 0.780 5.0 (2.0 to 8.0) 0.546 0.916
OCT macula 7.0 (3.5 to 8.8) 9.0 (7.3 to 10.0) 0.002


FFA/ICG 6.0 (4.0 to 8.0) 7.5 (3.3 to 9.0) 0.356
7. Facilitate comparison with previous results (compared to paper system)
VF 3.0 (2.0 to 6.8) 5.0 (2.0 to 7.0) 0.086 5.0 (3.0 to 9.0) 0.051 0.041

OCT RNFL 4.0 (2.0 to 7.0) 6.0 (3.0 to 8.0) 0.193 5.0 (3.0 to 9.0) 0.629 0.523
OCT macula 4.5 (2.0 to 7.8) 8.5 (6.3 to 10.0) 0.001


FFA/ICG 4.0 (3.0 to 6.8) 6.5 (3.0 to 9.0) 0.023

ePR indicates electronic patient record; FFA, fundal fluorescein angiography; ICG, indocyanine green angiography; OCT, optical coherence tomography, RNFL,
retinal nerve fiber layer; VF, visual field.




P < 0.05.



P < 0.05.


P < 0.05.

efficient in retrieving investigation report or images, making frequency of use of an individual system seemed to depend on
interpretation easier, or in facilitating comparison with previous system quality rather than image display quality. ePR, while
results, despite relatively high awareness of the multiple display having a lower score in information quality session, achieved
or comparison mode of HEYEX (75%). This study highlighted the highest scores in system quality compared to HEYEX and
some of the unique challenges faced by ophthalmologists when FORUM. The superior system stability may be explained by the
implementing PACS, despite its numerous evaluation in radiol- long duration of use of ePR in the public health system of nearly 2
ogy and physicians.5,19,20 decades. When the image viewing function was added to ePR, the
This study brought important insights about factors to con- adaptation needed was minimal, making it easiest to learn and use.
sider when choosing different types of PACS to implement in Together with fast login response time, system quality probably
ophthalmology. First, in a general ophthalmology setting, the contributed to the high usage of ePR, accounting for 80% of route

262 | https://journals.lww.com/apjoo © 2021 Asia-Pacific Academy of Ophthalmology.

 Asia-Pacific Journal of Ophthalmology  Volume 11, Number 3, May/June 2022 Use of Picture Archiving and Communication Systems

TABLE 4. Sensitivity Analysis for System Quality and Information Quality Between Different Subspecialties
TABLE 4A.
Items in HEYEX Scores by VRU Scores by Non-VRU P value
members (n ¼ 6) members (n ¼ 22)
System quality
Easy to learn 8.0, 6.0 to 9.3 6.0, 6.0 to 8.0 0.107
Downloaded from http://journals.lww.com/apjoo by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1A

Workstations easily available 8.0, 6.8 to 8.3 7.0, 6.0 to 8.0 0.356
Satisfactory response time to log in 8.0, 6.8 to 9.0 6.0, 4.0 to 7.0 0.020

Stable and seldom crash 6.5, 2.5 to 7.3 7.0, 5.0 to 8.0 0.318
Reduce patient identification error in filing the investigations 9.0, 8.0 to 10.0 8.0, 6.5 to 8.0 0.017

WnYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8K2+Ya6H515kE= on 03/04/2024

(compared to paper system)

Reduce patient identification error in file retrieval during 7.5, 7.0 to 8.3 8.0, 6.8 to 8.0 0.748
consultation (compared to paper system)
Information quality
1. Display all important information one looks for from the
reports/images
OCT macula 10.0, 9.8 to 10.0 8.0, 8.0 to 9.0 0.004


FFA/ICG 10.0, 8.5 to 10.0 8.0, 6.0 to 9.0 0.033

2. The display is of sufficient quality for interpretation
OCT macula 10.0, 9.8 to 10.0 9.0, 7.8 to 10.0 0.022

FFA/ICG 10.0, 7.5 to 10.0 8.0, 5.8 to 9.0 0.055
3. More efficient in retrieving investigation reports/images
(compared to paper system)
OCT macula 10.0, 9.0 to 10.0 8.0, 5.0 to 9.0 0.006


FFA/ICG 9.5, 7.5 to 10.0 5.5, 3.0 to 8.0 0.013

4. Make the interpretation easier
(compared to paper system)
OCT macula 10.0, 9.8 to 10.0 8.5, 6.0 to 9.0 0.005


FFA/ICG 10.0, 7.5 to 10.0 6.5, 3.0 to 8.0 0.012

5. Facilitate comparison with previous results
(compared to paper system)
OCT macula 10.0, 9.8 to 10.0 8.0, 5.8 to 9.0 0.002


FFA/ICG 10.0, 7.5 to 10.0 5.5, 3.0 to 8.0 0.007


TABLE 4B.
Items in FORUM Scores by GNO Scores by non-GNO P value
members (n ¼ 5) members (n ¼ 6)
System quality
Easy to learn 5.0, 2.5 to 7.5 6.0, 5.0 to 7.5 0.405
Workstations easily available 5.0, 3.0 to 6.5 5.5, 5.0 to 7.5 0.302
Satisfactory response time to log in 5.0, 4.0 to 6.5 7.5, 6.0 to 8.3 0.040

Stable and seldom crash 5.0, 2.5 to 7.0 6.5, 6.0 to 7.5 0.187
Reduce patient identification error in filing the 8.0, 6.0 to 9.0 7.0, 6.8 to 7.5 0.443
investigations (compared to paper system)
Reduce patient identification error in file retrieval 8.0, 6.0 to 8.0 7.5, 6.8 to 8.3 0.847
during consultation (compared to paper system)
Information quality
1. Display all important information one looks
for from the report/images
VF 9.0 (7.5 to 10.0) 8.0 (5.6 to 8.3) 0.133
OCT RNFL 8.0 (6.5 to 10.0) 6.5 (5.3 to 8.5) 0.264
2. The display is of sufficient quality for interpretation
VF 8.0 (8.0 to 9.5) 8.0 (5.6 to 8.3) 0.162
OCT RNFL 8.0 (7.5 to 9.0) 7.5 (5.8 to 9.3) 0.643
3. More efficient in retrieving investigation reports/images
(compared to paper system)
VF 6.0 (2.5 to 7.5) 5.5 (3.0 to 7.5) 1.000
OCT RNFL 6.0 (2.5 to 7.5) 8.5 (5.6 to 9.0) 0.096
4. Make the interpretation easier
(compared to paper system)
VF 7.0 (2.5 to 9.0) 5.5 (3.0 to 8.3) 0.783
OCT RNFL 7.0 (2.5 to 8.0) 5.0 (2.0 to 6.8) 0.713
5. Facilitate comparison with previous results
(compared to paper system)
VF 5.0 (2.5 to 9.5) 5.5 (3.0 to 9.3) 0.927
OCT RNFL 7.0 (2.5 to 9.5) 5.0 (3.0 to 6.8) 0.520
FFA indicates fundal fluorescein angiography; GNO, glaucoma/neuro-ophthalmology; ICG, indocyanine green angiography; OCT, optical coherence
tomography; RNFL, retinal nerve fiber layer; VF, visual field; VRU, vitreo-retina and uveitis.



P < 0.01.


P < 0.05 (asymptotic 2-tailed).

© 2021 Asia-Pacific Academy of Ophthalmology. https://journals.lww.com/apjoo | 263

 Lam et al. Asia-Pacific Journal of Ophthalmology  Volume 11, Number 3, May/June 2022

TABLE 5. Service Quality/Service, Benefit of PACS and Issues Encountered

TABLE 5A. Service Quality/Service Support
1. Need for technical support for the first 14 days of use (times/month) 0–2: 23 (82.1%)
2–4: 5 (17.9%)
2. Need for technical support is needed for the first 14 days of use (times/month) (n ¼ 27) 0–2: 25 (92.6%)
2–4: 2 (7.4%)
Downloaded from http://journals.lww.com/apjoo by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1A

3. Issues encountered while seeking technical support

a. Lack of time to seek technical support 14/28 (50.0%)
b. Technical support staff cannot be reached 5/28 (17.9%)
c. Assistant/other staff not being able to help 7/28 (25.0%)
WnYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8K2+Ya6H515kE= on 03/04/2024

4. Training provided before implementation is sufficient 7.0 (6.0 to 8.0)

5. Technical support provided before implementation is sufficient 7.0 (5.0 to 7.8)
6. Confident in using the system
a. ePR 9.5 (8.0 to 10.0)
b. HEYEX 7.0 (7.0 to 8.0)
c. FORUM (n ¼ 10) 5.0 (2.5 to 7.3)
TABLE 5B. Benefits of PACS
Shorten time needed for making clinical decisions compared with paper system 8/28 (28.6%)
Enhance quality of clinical decisions compared with paper system 13/28 (46.4%)
Enhance doctor-patient communication compared with paper system 9/28 (32.1%)
Enhance communication with colleagues compared with paper system 15/28 (53.6%)
Nil 5/28 (17.9%)
Other: Easier to review report by other clusters 1/28 (2.9%)

Issues Encountered When Using PACS

P value
P value P value (HEYEX
(ePR compared (ePR compared compared with
ePR HEYEX with HEYEX) FORUM with FORUM) FORUM)
Unable to identify 8/28 (28.6%) 4/28 (14.3%) 0.102 5/14 (35.7%) 0.564 0.157
location of investigations
Failure to log in 4/28 (14.3%) 8/28 (28.6%) 0.157 3/14 (21.4%) 0.655 0.317
Unacceptable login time 7/28 (25.0%) 14/28 (50.0%) 0.052 3/11 (21.4%) 0.414 0.102
Unacceptable loading time 6/28 (21.4%) 11/28 (39.3%) 0.132 4/14 (28.6%) 0.083 0.414
Poor resolution of 5/28 (17.9%) 0/28 (0.0%) 0.025
0/14 (0.0%) 0.317 1.000
investigations
Nil 9/28 (32.1%) 7/28 (25.0%) 7/14 (50.0%)
Other comments 0/28 (100.0%) 1/28 (2.9%): system 1/14 (7.1%):
“not responding” need training
for FORUM
Overall comments about electronic imaging system.
ePR indicates electronic patient records.
1/28 (3.6%): No marking of right or left eye for visual fields in e-systems making retrieval inefficient.
1/28 (3.6%): Prefer a comparison button so that visual fields can be displayed in chronological order rather than creating a layout by the user.


P < 0.05.

of access. Second, the pattern of use and perception of the system over paper/PDF format for VF/OCT RNFL interpretation. Over-
were subspecialty-dependent. In our hospital, OCT macula and all, the hybrid system of using both PDF format incorporated into
FA/ICG were performed mainly with Heidelberg SPECTRALIS, an existing ePR and DICOM format in platform-specific PACS
using the same platform as HEYEX, while OCT RNFL was (HEYEX and FORUM) gave us the advantage of catering for both
performed with Cirrus HD-OCT by Carl Zeiss Meditec, using busy general ophthalmology clinics, when response time is
the same platform as FORUM. In our study, the information crucial, and advanced subspecialty needs, where image quality
quality of OCT macula and FFA/ICG in HEYEX gained higher is more important. The choice of PACS would depend on the
scores among VRU doctors compared to other colleagues, dem- volume of practice and the degree of subspecialization.
onstrating a high demand for higher quality image display in Currently, there is no well-recognized tool for assessing the
aiding clinical diagnosis and treatment in vitreoretinal diseases. effectiveness of PACS. Evaluations of PACS were done through
On the other hand, FORUM did not achieve significantly higher heterogeneous methods, ranging from self-designed question-
scores among GNO doctors. Apart from limited viewing licences, naires5,19,21 to summarizing comments from professionals in online
another explanation was that only few patients had multiple VF or discussion groups,22 making direct comparison across studies dif-
OCT RNFL test results uploaded to the system, especially at the ficult. In our study, DeLone and McLean model of IS success was
early stage of implementation, limiting the availability of Guided chosen as it was well-validated and provided a comprehensive
Progression Analysis, which is the major advantage of FORUM framework to assess IS effectiveness,16 with past applications on

264 | https://journals.lww.com/apjoo © 2021 Asia-Pacific Academy of Ophthalmology.

 Asia-Pacific Journal of Ophthalmology  Volume 11, Number 3, May/June 2022 Use of Picture Archiving and Communication Systems

health IS, eg, electronic health record system.17 Although this model In conclusion, our results demonstrated overall positive
did not provide specific parameters under each domain, it allowed feedback from ophthalmologists as end-users of PACS in system
for a systematic approach to report and compare research work on IS quality and display of information, with major limitations being
success, and for study of interrelationships between domains,16 the inefficiency in the use of information and a lack of time in
which facilitated the comparison between various PACS in our accessing technical support. In particular, ophthalmologists work-
study. However, one should take caution that not all aspects of the ing in posterior segment had differential use pattern and feedback
Downloaded from http://journals.lww.com/apjoo by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1A

model were covered in our study. Net benefit, in particular, would on information display quality compared to those in other sub-
require a much more comprehensive survey covering the entire specialties, making subspecialty of an ophthalmology service a
workflow of archiving and retrieval of the ophthalmic imaging apart consideration for choosing PACS.
from the current end-user experience.
WnYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8K2+Ya6H515kE= on 03/04/2024

Ophthalmology involves substantial utilization of different

imaging modalities and evaluation of image-based data. With the REFERENCES
rapid advancement in artificial intelligence, deep learning, and big 1. Tong CKS, Wong ETT. Picture Archiving and Communication System for
data, which have fueled mankind’s fourth industrial revolution,23 it Public Healthcare. Pennsylvania, US: IGI Global; 2011.
is now possible to harness the vast volume of ocular images for 2. Lemke HU. A Network of Medical Work Stations for Integrated Word and
analysis to yield highly accurate diagnostic information.24–29 Nev- Picture Communication in Clinical Medicine, Technical Report. Berlin:
ertheless, to meaningfully utilize big data in ophthalmology, in Technical University; 1979.
which images play an essential role, developing the necessary
3. Meyer-Ebrecht D. Picture archiving and communication systems (PACS)
softwares and hardwares to handle, share, and analyze image-based
for medical application. Int J Biomed Comput. 1994;35:91–124.
data from different sources would be crucial.30 Understanding the
doi:10.1016/0020-7101(94)90061-2.
strengths and deficits of the different PACS from the perspective of
ophthalmologists would be pivotal in aiding developers to stan- 4. Madapana N, Gonzalez G, Rodgers R, et al. Gestures for picture archiving and

dardize, refine, and optimize such systems to efficiently facilitate communication systems (PACS) operation in the operating room: is there any

patient care and research to fit the needs of the information age. standard? PLoS One. 2018;13:e0198092. doi:10.1371/journal.pone.0198092.

Our study had several limitations. First, the small sample size 5. Alalawi ZM, Eid MM, Albarrak AI. Assessment of picture archiving and
and single-center design would limit the generalizability of our communication system (PACS) at three of ministry of health hospitals in
results to ophthalmology services worldwide. However, the inclu- Riyadh region - content analysis. J Infect Public Health. 2016;9:713–724.
sion of 3 types of PACS and sensitivity analysis among different doi:10.1016/j.jiph.2016.09.004.
subspecialty members, which was made possible as our center 6. Mackinnon AD, Billington RA, Adam EJ, et al. Picture archiving and
provided high-volume service in both general and subspecialized communication systems lead to sustained improvements in reporting times
ophthalmology, would enhance the relevance of our results to a and productivity: results of a 5-year audit. Clin Radiol. 2008;63:796–804.
broader range of ophthalmic practices. Second, our survey doi:10.1016/j.crad.2007.12.015.
assessed only respondents’ subjective estimation of events, which 7. Mwanza JC, Budenz DL. New developments in optical coherence
would lead to recall bias. More objective evaluations including tomography imaging for glaucoma. Curr Opin Ophthalmol. 2018;29:121–
measurement of consultation time both pre-implementation and 129. doi:10.1097/ICU.0000000000000452.
post-implementation of PACS may be considered in future stud- 8. Al-latayfeh MM, Sun JK, Aiello LP. Ocular coherence tomography and
ies. Although login time and image loading time were not diabetic eye disease. Semin Ophthalmol. 2010;25:192–197. doi:10.3109/
measured for each respondent, an estimation by 2 authors (TL 08820538.2010.518107.
and MW) showed results consistent with the questionnaire
9. Hospital Authority. Hospital Authority Major Statistics. 2019.
response (average login time: ePR: 2.9 s, HEYEX: 10.9 s,
FORUM: 4.1 s; average loading time of all types of images 10. Hospital Authority. HA Quality and Safety Annual Report. 2019.

available in the platform: ePR: 0.8 s, HEYEX: 1.8 s, FORUM: 11. Balyen L, Peto T. Promising artificial intelligence-machine learning-deep
1.7 s). However, one should be cautious that numerous factors, for learning algorithms in ophthalmology. Asia Pac J Ophthalmol (Phila).
example, hardware, image size, etc, may affect the direct appli- 2019;8:264–272. doi:10.22608/AP0.2018479.
cation of these results to PACS installed in other settings. Third, 12. Huang HK. From PACS to web-based ePR system with image distribution
our study did not assess the benefits or costs of PACS from the for enterprise-level filmless healthcare delivery. Radiol Phys Technol.
health system and hospital management perspective, which is a 2011;4:91–108. doi:10.1007/s12194-011-0122-5.
crucial component for understanding the cost-effectiveness of 13. Chantler C, Clarke T, Granger R. Information technology in the English National
PACS in health systems, and therefore informing policy planning. Health Service. Jama. 2006;296:2255–2258. doi:10.1001/jama.296.18.2255.
The cost and expertise required in setting up the systems, which 14. Pandit RR, Boland MV. Impact of digital imaging and communications in
may affect the ophthalmologists’ perception of the system, were medicine workflow on the integration of patient demographics and ophthalmic
not evaluated in this study as the systems in HKEH were set up by test data. Ophthalmology. 2015;122:227–232. doi:10.1016/j.ophtha.2014.08.036.
respective management and information technology teams and
15. (MITA) TMITA. DICOM Strategic document. Update on March 8, 2017..
information system personnel. Fourth, factors affecting the accep-
Available from https://www.dicomstandard.org Accessed May 3, 2021.
tance rate of PACS among staff were not discussed in this study,
as paper printouts for images were no longer available soon after 16. DeLone W, McLean E. The DeLone and McLean model of information

implementation of PACS, making use of the latter compulsory. systems success: a ten-year update. J Manag Inf Syst. 2003;19:9–30.

Lastly, built-in functions of PACS are numerous and being doi:10.1080/07421222.2003.11045748.

upgraded from time to time, and the results of our survey may 17. Bossen C, Jensen LG, Udsen FW. Evaluation of a comprehensive EHR
not be applicable to future versions of the systems. based on the DeLone and McLean model for IS success: approach, results,

© 2021 Asia-Pacific Academy of Ophthalmology. https://journals.lww.com/apjoo | 265

 Lam et al. Asia-Pacific Journal of Ophthalmology  Volume 11, Number 3, May/June 2022

and success factors. Int J Med Inform. 2013;82:940–953. doi:10.1016/j. 24. Abramoff MD, Lou Y, Erginay A, et al. Improved automated detection of
ijmedinf.2013.05.010. diabetic retinopathy on a publicly available dataset through integration of
18. Khajouei R, Jahromi ME, Ameri A. Challenges of implementing picture archiving deep learning. Invest Ophthalmol Vis Sci. 2016;57:5200–5206. doi:10.1167/
and communication system in multiple hospitals: perspectives of involved staff iovs.16-19964.
and users. J Med Syst. 2019;43:182. doi:10.1007/s10916-019-1319-0. 25. Breusegem C, Fieuws S, Stalmans I, et al. Agreement and accuracy of
19. Carlin L, Smith H, Henwood F, et al. Double vision: an exploration of nonexpert ophthalmologists in assessing glaucomatous changes in serial
Downloaded from http://journals.lww.com/apjoo by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1A

radiologists’ and general practitioners’ views on using picture archiving and stereo optic disc photographs. Ophthalmology. 2011;118:742–746.
communication systems (PACS). Health Informatics J. 2010;16:75–86. doi:10.1016/j.ophtha.2010.08.019.
doi:10.1177/1460458210361935. 26. Goh JHL, Lim ZW, Fang X, et al. Artificial intelligence for cataract
WnYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8K2+Ya6H515kE= on 03/04/2024

20. Arora D, Mehta Y. Use of picture archiving and communication system for detection and management. Asia Pac J Ophthalmol (Phila). 2020;9:88–95.
imaging of radiological films in cardiac surgical intensive care unit. J Anaesthesiol doi:10.1097/01.Apo.0000656988.16221.04.
Clin Pharmacol. 2014;30:447–448. doi:10.4103/0970-9185.137306. 27. He M, Li Z, Liu C, et al. Deployment of artificial intelligence in real-world
21. Abbasi R, Sadeqi Jabali M, Khajouei R, et al. Investigating the satisfaction practice: opportunity and challenge. Asia Pac J Ophthalmol (Phila).
level of physicians in regards to implementing medical Picture Archiving 2020;9:299–307. doi:10.1097/apo.0000000000000301.
and Communication System (PACS). BMC Med Inform Decis Mak. 28. Huang SS. Future Vision 2020 and beyond-5 critical trends in eye research.
2020;20:180. doi:10.1186/s12911-020-01203-0. Asia Pac J Ophthalmol (Phila). 2020;9:180–185. doi:10.1097/
22. Alhajeri M, Shah SGS. Limitations in and solutions for improving the apo.0000000000000299.
functionality of picture archiving and communication system: an 29. Ruamviboonsuk P, Cheung CY, Zhang X, et al. Artificial intelligence in
exploratory study of PACS professionals’ perspectives. J Digit Imaging. ophthalmology: evolutions in Asia. Asia Pac J Ophthalmol (Phila).
2019;32:54–67. doi:10.1007/s10278-018-0127-2. 2020;9:78–84. doi:10.1097/01.APO.0000656980.41190.bf.
23. Ting DSW, Pasquale LR, Peng L, et al. Artificial intelligence and deep 30. Cheng CY, Soh ZD, Majithia S, et al. Big data in ophthalmology. Asia Pac
learning in ophthalmology. Br JOphthalmol. 2019;103:167–175. J Ophthalmol (Phila). 2020;9:291–298. doi:10.1097/
doi:10.1136/bjophthalmol-2018-313173. apo.0000000000000304.

266 | https://journals.lww.com/apjoo © 2021 Asia-Pacific Academy of Ophthalmology.