Abstract

The English version of the online encyclopedia, Wikipedia, has been recently reported to be the prominent source of online health information. However, there is little information concerning the quality of information found in Wikipedia. Therefore, we created a questionnaire asking for scope, completeness, and accuracy of information found on osteosarcoma. Three independent observers tested the English version of Wikipedia, as well as the patient version and the health professional version of the US National Cancer Institute (NCI) website. Answers were verified with authoritative resources and international guidelines. The results of our study demonstrate that the quality of osteosarcoma-related information found in the English Wikipedia is good but inferior to the patient information provided by the NCI. Therefore, non-peer-reviewed commonly used websites offering health information, such as Wikipedia, should include links to more definitive sources, such as those maintained by the NCI and professional international organizations on healthcare treatments. Furthermore, frequent checks should make sure such external links are to the highest quality and to the best-maintained aggregate sites on a given healthcare topic.

Introduction

In a recent issue of JAMIA,1 Laurent and Vickers have reported that the English version of the online encyclopedia, Wikipedia, is a prominent source of online health information. However, there is little information concerning the quality of health information found in Wikipedia.2 For example, accurate and reliable patient information plays a crucial role in the multidisciplinary treatment of malignancies. Access to ‘best evidence’ helps to promote compliance of the patients and their relatives with often long-lasting and stressful treatments. Furthermore, the amount and quality of information regarding a specific tumor-type and its treatment might influence patients' decisions to participate in clinical trials. Available information may reduce instances of denial of treatment that can even diminish a patient's chance of survival. Patients and their families often turn to internet sources of health information to increase their knowledge about specific conditions and also to counter-check the information they receive from their doctors.1

Background

Therefore, the authors decided to compare the completeness and accuracy of information on osteosarcoma found in Wikipedia to other available online sources. We chose osteosarcoma as it is the most common type of malignant bone tumor with a reported annual incidence rate of 0.42 per 100, affecting mainly children and young adults.3 Besides factors like location, size, and presence or absence of metastases, survival mainly depends on an early diagnosis and a correct multidisciplinary treatment, including surgery and chemotherapy. Similar to the test approach used by Clauson et al,2 we created a questionnaire comprised of 20 questions related to scope, completeness, and accuracy of information found on osteosarcoma (table 1).

Table 1

Osteosarcoma questionnaire presenting the points for each answer for each of the three different websites: (1) the English version of Wikipedia on osteosarcoma; (2) the patient version of the National Cancer Institute (NCI) website; and (3) the health professional version of the NCI website

No Question Wikipedia NCI patient NCI professional 
What is the incidence of osteosarcoma per 100 inhabitants? 
Should a biopsy be performed? 
What is a biopsy? 
Name three helpful chemotherapeutic agents 
Should radiotherapy normally be applied? 
What kind of surgery should be performed? (margins) 
Are amputations necessary in a large number of cases? 
What does staging imply? 
Is a follow-up necessary? If yes, name 2 diagnostic procedures 
10 What is the 5 or 10-y prognosis? 
11 Prognosis depends on several factors, name 5 
12 Which age group is most affected? 
13 Might metastases occur? If yes, what are metastases? 
14 Name three histological subtypes 
15 What symptoms might indicate the presence of a bone tumor (name 2) 
16 What is the main localization of osteosarcoma? 
17 What late effects are possible after successful treatment? (name 3) 
18 Do you find web-links to study centers (EURAMOS)? 
19 What are clinical trials? 
20 How can you find clinical trials in your area? 
 Total 33 40 50 
No Question Wikipedia NCI patient NCI professional 
What is the incidence of osteosarcoma per 100 inhabitants? 
Should a biopsy be performed? 
What is a biopsy? 
Name three helpful chemotherapeutic agents 
Should radiotherapy normally be applied? 
What kind of surgery should be performed? (margins) 
Are amputations necessary in a large number of cases? 
What does staging imply? 
Is a follow-up necessary? If yes, name 2 diagnostic procedures 
10 What is the 5 or 10-y prognosis? 
11 Prognosis depends on several factors, name 5 
12 Which age group is most affected? 
13 Might metastases occur? If yes, what are metastases? 
14 Name three histological subtypes 
15 What symptoms might indicate the presence of a bone tumor (name 2) 
16 What is the main localization of osteosarcoma? 
17 What late effects are possible after successful treatment? (name 3) 
18 Do you find web-links to study centers (EURAMOS)? 
19 What are clinical trials? 
20 How can you find clinical trials in your area? 
 Total 33 40 50 

EURAMOS, European and American Osteosarcoma Study Group.

Testing wikipedia

Three independent observers (WM and MG, two surgeons specialized in musculoskeletal tumor surgery, and JF, a medical student) tested the English version of Wikipedia on osteosarcoma (date: 3 April 2009), as well as the patient version and the health professional version of the National Cancer Institute's (NCI) website as ‘official’ reference websites. Answers (scores on a 0–3 scale) were discussed with a member of the German board for guidelines in musculoskeletal tumor surgery (AL) and, furthermore, verified with authoritative resources4 and international guidelines.5 Data were compiled and analyzed with the software SPSS version 17.0. Group comparisons were performed using Mann–Whitney U test. We considered p-values smaller than 0.05 as significant.

The professional version of the NCI scored best with 50 of 60 possible points (mean 2.5±0.83 SD) compared to the patient version of the NCI with 40 points (mean 2±1.38 SD) and to the Wikipedia site with 33 points (mean 1.65±1.39 SD). For example, important information on clinical studies or possible study centers (European and American Osteosarcoma Study Group (EURAMOS)) was missing on Wikipedia. Only the difference between the NCI professional version and Wikipedia was significant (p=0.039). However, all three reviewers preferred Wikipedia when asked for the ease of use to find patient-related information and an explanation of the page's oversight.

In our limited study we were able to demonstrate that the quality of osteosarcoma-related information found in English Wikipedia is good but inferior to the patient information provided by the NCI. In our opinion, more definitive sources of medical information, such as those discussed for osteosarcoma, should, therefore, be checked regularly by medical specialists as well as patients and their families.

Callis et al called improving Wikipedia an ‘educational opportunity and professional responsibility’, stating that revision of Wikipedia entries should be incorporated in undergraduate courses and annual meetings of professional societies.6 Eysenbach and Diepgen suggested that doctors, medical societies, and associations should critically appraise internet information and act as decentralized ‘label services' to rate the value and trustworthiness of information found on the internet.7 In particular, the molecular-biology scientific community has taken this opportunity and has so far incorporated several specially labeled molecular biology databases into Wikipedia – RNA WikiProject, WikiProteins, GeneWiki, and Proteopedia. It is not clear whether primary information providers, such as the US National Cancer Institute, share the responsibility to ‘publish in duplicate’ on their own definitive ‘in-house’ web sites as well as on public sites such as Wikipedia. Because Wikipedia and similar sites are easy to find and easy to read and understand (as was found in our informal study), it is important that people maintaining these sites include links to more definitive and comprehensive aggregate and summary web resources, such as the NCI's material for patients and professionals regarding various types of cancer.

As Wikipedia seems to be the first choice for health-related information found on the internet,8 even for junior physicians, the responsibility for correctness is high. Although we are aware of the limitations of this study—small sample size in items tested and in reviewers, as well as the creation of the questions without external review board—we think that its results might initiate some discussion in the relevant societies, and also maybe a larger international study on various bone and soft tissue tumors described in Wikipedia.

Conclusion

In conclusion, our study shows that the quality of osteosarcoma-related information found in the English Wikipedia is good but inferior to the patient information provided by the NCI. In particular, in multidisciplinary treated tumors like osteosarcoma, accurate and complete information found on Wikipedia might influence patients' compliance with treatment. Therefore, non-peer-reviewed commonly used websites offering health information, such as Wikipedia, should include links to more definitive sources such as those maintained by the NCI and professional international organizations on healthcare treatments. Furthermore, frequent checks should make sure such external links are to the highest quality and to the best-maintained aggregate sites on a given healthcare topic.

Acknowledgements

I would like to thank the JAMIA editorial staff for their helpful assistance.

Competing interests

None.

Provenance and peer review

Not commissioned; externally peer reviewed.

References

1
Laurent
MR
Vickers
TJ
.
Seeking health information online: does Wikipedia matter?
J Am Med Inform Assoc
 
2009
;
16
:
471
9
.
2
Clauson
KA
Polen
HH
Boulos
MN
et al
.
Scope, completeness, and accuracy of drug information in Wikipedia
.
Ann Pharmacother
 
2008
;
42
:
1814
21
.
3
Damron
TA
Ward
WG
Stewart
A
.
Osteosarcoma, chondrosarcoma, and Ewing's sarcoma: National Cancer Data Base Report
.
Clin Orthop Relat Res
 
2007
;
459
:
40
7
.
4
Raymond
AK
Ayala
AG
Knuutila
S
.
Conventional osteosarcoma
. In:
Fletcher
DM
Unni
KK
Mertens
F
, eds.
World Health Organization classification of tumours: pathology and genetics of tumors of soft tissue and bone
 .
Lyon
:
IARCPress
,
2002
:
264
70
.
5
Dorfman
HD
Czerniak
B
Kotz
R
et al
.
WHO classification of tumours of bone: Introduction
. In:
Fletcher
DM
Unni
KK
Mertens
F
, eds.
World Health Organization classification of tumours. Pathology & genetics. Tumours of soft tissue and bone
 .
Lyon
:
IARC Press
,
2000
:
227
32
.
6
Callis
KL
Christ
LR
Resasco
J
et al
.
Improving Wikipedia: educational opportunity and professional responsibility
.
Trends Ecol Evol
 
2009
;
24
:
177
9
.
7
Eysenbach
G
Diepgen
TL
.
Towards quality management of medical information on the internet: evaluation, labelling, and filtering of information
.
BMJ
 
1998
;
317
:
1496
500
.
8
Hughes
B
Joshi
I
Lemonde
H
et al
.
Junior physician's use of Web 2.0 for information seeking and medical education: A qualitative study
.
Int J Med Inform
 
2009
;
78
:
645
55
.

Comments

0 Comments