Printer Friendly

Evaluation of Internet Information about Lingual Orthodontics Using DISCERN and JAMA Tools.


The internet is a source of information that is increasingly used by both health professionals and patients (1). According to statistics released by the Turkish Statistical Institute, in 2016, 54.9% of individuals in the 16-74 age group have been using computers, and 61.2% the internet. In the first 3 months of 2016, internet users used it predominantly (82.4%) for creating profiles on social media, sending messages, and sharing photos. These tasks were closely followed by watching videos (74.5%); reading online news, newspapers, or magazines (69.5%); searching for health-related information (65.9%); searching for information about goods and services (65.5%); and listening to music (63.7%). The percentage of internet users surfing almost every day or at least once a week was 94.9% (2). The search for health-related information was reported to be 94.9% among regular users by Demirel et al. (3). According to the results of their research, the internet is preferred because it is an easy, cheap, and fast way of accessing information; 30.4% of internet users make their health-related decisions based on the internet; and they also use this information to communicate with their physicians.

Traditionally, a person in need of orthodontic treatment is informed directly about his or her own malocclusion following the examination by the orthodontist. However, today, most of the patients inform themselves through the internet, even before going to the doctor. For these reasons, the quality, reliability, and accuracy of information on the web is critical. To help both clinicians and patients to choose quality websites on health-related information, validation tools were developed that can assess various properties of internet sites (4-6). Quality Criteria for Consumer Health Information (DISCERN), Journal of American Medical Association (JAMA) benchmarks, LIDA (Minervalidation Inc.), Health on the Net Code of Conduct (HONCode) are some of those. Up to now, the number of studies on internet-based information on orthodontics has been limited (7-11). Measurement tools, search engines, and keywords used in these studies vary.

The increase in the number of adults opting for orthodontic treatment can be traced back to various factors, such as the increase in aesthetic awareness, improved appearance of fixed orthodontic devices, and increased social acceptance of fixed orthodontic devices (12, 13). Lingual orthodontic treatment usually is preferred by adult patients with high aesthetic expectations. As treatment mechanics continue to develop, the interest in this field continues to increase, although only slowly, due to its technical difficulties and high cost (14, 15). Because of patients increasing demand of lingual orthodontics, and because, to our knowledge, there are no studies about internet information on lingual orthodontics, the purpose of this study is to evaluate the quality of information on lingual orthodontics on the internet using DISCERN and JAMA scales.


An internet search was conducted on March 6th, 2017, using popular search engines in Turkey: GoogleTM ( com), bingTM (, YAHOO![R] (, and Yandex[R] ( (16). The terms "lingual ortodonti" (lingual orthodontics), "gorunmeyen braketler" (invisible braces), and "gorunmeyen teller" (invisible wires) were used as keywords, because these were phrases most often used by patients in our experience. The top 10 websites for each keyword and search engine were evaluated. Scientific articles and orthodontic product websites were not included. After excluding duplicates and irrelevant websites (Figure 1), the remaining 58 websites were scored by a single examiner (HKO-orthodontist). The websites included in the study were evaluated using the DISCERN tool and JAMA benchmarks. The website type, presentation type, the profession of the author, and target group were also recorded.


The DISCERN tool was developed by Charnock et al. (17) for the health field in 1998 and has been translated into Turkish by Gokdogan et al. (18). DISCERN consists of 16 questions (graded 1-5) and three parts: reliability (Questions 1-8), quality information on treatment choices (Questions 9-15), and overall score (Question 16).

The DISCERN manual contains detailed information for each question, as well as instructions and examples to make the evaluation easy. According to this tool, considering the total average scores, websites were divided into 5 groups as follows: score between 16 and 26 is very poor, score between 27 and 38 is poor, score between 39 and 50 is fair, score between 51 and 62 is good, and score higher than 63 is excellent.

JAMA Benchmarks

The JAMA benchmarks were published as a suggestion for basic quality standards for internet information on health care by Silberg, Lundberg, and Musacchio in 1997 (19). It evaluates four key features that must be clearly visible on a website:

Authorship (Author): Writers and contributors should be informed about their linkages and subject qualifications.

Attribution: References and references for all content should be clearly listed, and copyright information should be included.

Disclosure: The potential conflict of interest arising out of the website's ownership, sponsorship, advertising, insurance liability, commercial financing, or support must be clearly and fully disclosed.

Currency: The dates on which the content was uploaded and updated should be specified.

Care should be taken to ensure that each criterion is clearly stated when the assessment is made.

Statistical Analysis

Statistical data processing was performed using Microsoft Excel Version 2016 (MS Excel 2016). Descriptive analysis such as mean and frequency was calculated.


From the 120 websites found, 62 were excluded (41 duplicated, 21 irrelevant) (Figure 1). The authors of the remaining 58 sites were orthodontists (48%), dentists (5%), and non-disclosed authors (47%). The total DISCERN score of the 58 websites was poor (average score 28.9). No website has reached excellent, or good score. More than half of the websites were scored as poor, or very poor (64%) (Table 1).

According to presentation type, 91% of the websites were in text form, and 9% of the websites were in video form. According to website type, 74% of the websites were in the form of advertisement, and 26% of the websites were prepared for information purposes only (Table 2).

When using the keyword "gorunmeyen teller" (invisible wires), two websites were explaining only aligner treatment, four websites were explaining lingual orthodontics, but the photos were about aligner treatment, and one website was explaining lingual orthodontics, but the video in the website was about buccal braces. When using "gorunmeyen braketler" (invisible braces), one video-type website was about ceramic braces, and one website was explaining lingual orthodontics, but the photos were about aligner treatment.

Table 3 shows how the websites performed for each question; it shows the average scores out of five. The question about "achieving the aims" (Question 2) scored highest; followed by the question about "clarity regarding the aims" (Question 1). The lowest scoring questions were "if no treatment was used" (Question 12) and "clarity regarding the sources of information used to compile the publication" (Question 4).

Assessment according to JAMA benchmarks revealed no website that met all JAMA benchmarks. The principle of disclosure was adhered to most frequently, while the principle of attribution was the most poorly adhered (Table 4).


This is the first study to evaluate the quality of information related to lingual orthodontics in Turkish on the internet. This study has been designed in view of the fact that information about lingual orthodontics on the internet is likely to be inadequate or incorrect.

The top 10 websites were evaluated, considering that the keywords entered in each search engine came from a large number of internet sites, but the patients often look the first page (20). The scope of the internet is very broad, and a search yields millions of results, but naturally, the user only displays some of them.

One-third of the websites were duplicated websites showing that different search engines and the three different keywords do not produce vastly different results. The keywords were chosen presuming what lay people might employ when searching the internet for lingual orthodontics. But results showed also other treatment options under the name of lingual orthodontics. This confusion might be because these keywords could also be preferred for ceramic braces or aligner treatment. Using the Google Trends application to determine keywords might help to get popular keywords used for search in Turkey and in the world. It might be argued that different keywords might have produced different results for the first 10 websites. The fact that one-sixth of the initial websites were irrelevant shows either that the keywords chosen did not pinpoint lingual orthodontic treatment alone, or that website providers did not bother with quality of the content.

Internet studies on orthodontics have reported that the quality of internet information is variable. Patel and Cobourne (7) used LIDA and FRES tools with the keyword "orthodontic extraction", and Google[TM] and YAHOO![R] search engines, and found that their reliability of the websites was inadequate. Parekh and Gill (8) used LIDA tool and GDC criteria for the keyword "orthodontic practice" and three different search engines (UK-based sites), and reported that websites generally do not comply with ethical rules and are not sufficiently reliable.

Verhoef et al. (9) used LIDA and FRES tools with the keywords "cleaning braces, brushing braces, oral hygiene and braces" and GoogleTM, YAHOO![R], and bingTM search engines, and found that the quality was low. Patel and Cobourne (10) used DISCERN, LIDA, and GDC criteria with the keyword "orthodontic braces" using the Google [TM] search engine and stated that many websites do not comply with ethical rules, and the quality of information varies. McMorrow and Millett (11) used DISCERN, JAMA, FRES, LIDA, and HONCODE tools with the keyword "adult orthodontics" using Google[TM], YAHOO! [R], and bingTM search engines, and reported that informative websites were limited and of fair quality. Our study showed that the information on the internet related to lingual orthodontics is poor, parallel to the above-mentioned studies.

In lingual orthodontics, customized lingual braces and wire systems like Incognito[TM] and Harmony[R], as well as fabricated lingual braces and wires, are used. Websites do not adequately describe and compare these treatment options.

When websites are being prepared, reference sources should be specified (attribution), and the date on which the information is uploaded and updated (currency) should be explicitly included on the website. According to JAMA benchmarks, the biggest shortcoming among websites were these two criteria. Only four of the websites met the criteria of reference, and two of them met the criteria of currency.

Sometimes different instruments can evaluate the same features. For example, DISCERN tool's 4th and 7th questions are parallel to the 2nd JAMA benchmark, and the 5th question is parallel to the 4th benchmark. However, JAMA is mainly evaluating the reliability of websites, whereas the DISCERN instrument is evaluating the quality of information, meaning reliability, and accuracy of content. Even in websites that met three benchmarks of JAMA in this research, DISCERN tool average score was low. For this reason, using more than one tool was considered to be useful for the objective evaluation of websites.

There are geographical and timewise limitations of research about the internet. Because the search was performed in the Turkish language, the research had validity only in this geography. Because the search was done in March 2017, new websites may have appeared, some may have been updated, or have been out of view. Another limitation is that only the top 10 websites for each keyword and search engine were evaluated, presuming this is the predominant behavior of the common internet user.

For this reason, it would be advisable to have conduct such research regularly by the relevant associations or organizations (e.g., Turkish Orthodontic Society [TOD]) and publish the results. Since TOD's page is not among the top 10 sites in the internet, it might be beneficial in TOD's and public interest to have prepared a web page with detailed and objective information on lingual orthodontics.

In order to provide quality health care services in the future, the knowledge of orthodontics needs to be improved continuously. Orthodontists should help patients get accurate and reliable information by directing them to evidence-based educational materials on the internet.


The quality of information on the internet related to lingual orthodontic treatment is poor. In the light of these results, patients should be cautious about trusting information on the internet on lingual orthodontics. Orthodontists should use these or similar tools as a guide, when creating an informative website.

Ethics Committee Approval: As the paper does not deal with humans or any material previously collected from humans, no ethical approval was taken.

Peer-review: Externally peer-reviewed.

Author Contributions: Concept - H.K.O., A.A.D.; Design - H.K.O., A.A.D.; Supervision - H.K.O., A.A.D.; Resources - H.K.O., A.A.D.; Materials - H.K.O., A.A.D.; Data Collection and/or Processing - H.K.O., A.A.D.; Analysis and/or Interpretation - H.K.O., A.A.D.; Literature Search - H.K.O., A.A.D.; Writing Manuscript - H.K.O., A.A.D.; Critical Review - H.K.O., A.A.D.

Conflict of Interest: The authors have no conflicts of interest to declare.

Financial Disclosure: The authors declared that this study has received no financial support.


(1.) Anderson JG, Rainey MR, Eysenbach G. The impact of Cyber-Healthcare on the physician-patient relationship. J Med Syst 2003; 27: 67-84. [CrossRef]

(2.) Turkiye Istatistik Kurumu (

(3.) Demirel M, Tekin A, ozbek S, Kaya E. E-Saglik kapsaminda internet kullanicilarinin saglik web sitelerini kullanma durumu uzerine bir arastirma. Mehmet Akif Ersoy Universitesi Bilimsel Arastirma Projeleri Komisyonu (Proje No: 0016-NAP-07 ). 2008.

(4.) Eysenbach G, Powell J, Kuss O, Sa ER. Empirical studies assessing the quality of health information for consumers on the World Wide Web: systematic review. JAMA 2002; 287: 2691-700. [CrossRef]

(5.) Nghiem AZ, Mahmoud Y, Som R. Evaluating the quality of internet information for breast cancer. Breast 2016; 25: 34-7. [CrossRef]

(6.) Livas C, Delli K, Ren Y. Quality evaluation of the available Internet information regarding pain during orthodontic treatment. Angle Orthod 2013; 83: 500-6. [CrossRef]

(7.) Patel U, Cobourne MT. Orthodontic extractions and the Internet: quality of online information available to the public. Am J Orthod Dentofacial Orthop 2011; 139: 103-9. [CrossRef]

(8.) Parekh J, Gill DS. The quality of orthodontic practice websites. Br Dent J 2014; 216: 21. [CrossRef]

(9.) Verhoef WA, Livas C, Delli K, Ren Y. Assessing the standards of online oral hygiene instructions for patients with fixed orthodontic appliances. J Am Dent Assoc 2015; 146: 310-7. [CrossRef]

(10.) Patel A, Cobourne MT. The design and content of orthodontic practise websites in the UK is suboptimal and does not correlate with search ranking. Eur J Orthod 2015; 37: 447-52. [CrossRef]

(11.) McMorrow SM, Millett DT. Adult orthodontics: a quality assessment of Internet information. J Orthod 2016; 43: 186-92. [CrossRef]

(12.) Nattrass C, Sandy JR. Adult orthodontics-a review. Br J Orthod 1995; 22: 331-7. [CrossRef]

(13.) Zachrisson BU. Global trends and paradigm shifts in clinical orthodontics. World J Orthod 2005; 6: 3-7.

(14.) Soldanova M, Leseticky O, Komarkova L, Dostalova T, Smutny V, Spidlen M. Effectiveness of treatment of adult patients with the straightwire technique and the lingual two-dimensional appliance. Eur J Orthod 2012; 34: 674-80. [CrossRef]

(15.) Miyawaki S, Yasuhara M , Koh Y. Discomfort caused by bonded lingual orthodontic appliances in adult patients as examined by retrospective questionnaire. Am J Orthod Dentofacial Orthop 1999; 115: 83-8. [CrossRef]

(16.) Search engines in Turkey. Available from: http://www. Accessed on November 17, 2017.

(17.) Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health 1999; 53: 105-11. [CrossRef]

(18.) Gokdogan F, Kir E, ozcan A, Cerit B, Yildirim Y ve Akbal S. Egitim kitapciklari guvenilir mi? 2. Uluslararasi 9. Ulusal Hemsirelik Kongresi; 2003; Antalya, Turkey.

(19.) Silberg WM, Lundberg GD, Musacchio RA. Assessing, controlling, and assuring the quality of medical information on the Internet: Caveant lector et viewor e let the reader and viewer beware. JAMA 1997; 277: 1244-5. [CrossRef]

(20.) van Deursen AJ, van Dijk JA. Using the Internet: skill related problems in users' online behavior. Interacting with Computers 2009; 21: 393-402. [CrossRef]

Hatice Kubra Olkun 1D, Arzu Ari Demirkaya 1D

Department of Orthodontics, Okan University School of Dentistry, Istanbul, Turkey

Address for Correspondence: Hatice Kubra Olkun, Okan University School of Dentistry, Istanbul, Turkey


Received: 15 September 2017

Accepted: 25 December 2017

Available Online Date: 11 April 2018

ORCID IDs of the authors: H.K.O. 0000-0001-5957-5819; A.A.D. 0000-0002-6039-5433.

DOI: 10.5152/TurkJOrthod.2018.17042

Cite this article as: Olkun HK, Ari Demirkaya A. Evaluation of Internet Information about Lingual Orthodontics Using DISCERN and JAMA Tools. Turk J Orthod 2018; 31: 50-4.
Table 1. DISCERN and JAMA scores according to profession of the authors

                                   Total (n=58)  Orthodontist (n=28)

Total DISCERN score (16-80)
16-26 (very poor)                      26              11
27-38 (poor)                           23              10
39-50 (fair)                            9               7
51-62 (good)                            0               0
63-80 (excellent)                       0               0
Average DISCERN score                  28.9            30.5
Average number of JAMA benchmarks       1.36            1.8
satisfied (0-4)

                                   Dentist (n=3)  No Author Disclosed

Total DISCERN score (16-80)
16-26 (very poor)                        1             14
27-38 (poor)                             2             11
39-50 (fair)                             0              2
51-62 (good)                             0              0
63-80 (excellent)                        0              0
Average DISCERN score                   30             27.1
Average number of JAMA benchmarks        2.3            0.8
satisfied (0-4)

Table 2. DISCERN and JAMA scores according to information type and
website type

                   Presentation Type    Website Type
                   Video  Text        Information  Advertisement
                   (n=5)  (n=53)      (n=15)       (n=43)

score (16-80)
16-26 (very poor)    4     22           8           18
27-38 (poor)         0     23           5           18
39-50 (fair)         1      8           2            7
51-62 (good)         0      0           0            0
63-80 (excellent)    0      0           0            0
Average score       24.8   29.3        28.5         29.0
Average number of    1.2    1.36        1            1.49
JAMA benchmarks
satisfied (0-4)

Table 3. Average score per DISCERN question amongst all websites

  DISCERN Questions                                       Mean Score

1   Are the aims clear?                                   2.5
2   Does it achieve its aims?                             2.93
3   Is it relevant?                                       2.25
4   Is it clear what sources of information were used     1.12
    to compile the publication (other than the author
    or producer)?
5   Is it clear when the information used or reported     1.36
    in the publication was produced?
6   Is it balanced and unbiased?                          1.81
7   Does it provide details of additional sources of      1.45
    support and information?
8   Does it refer to areas of uncertanity?                1.78
9   Does it describe how each treatment works?            2.09
10  Does it describe the benefits of each treatment?      2.36
11  Does it describe the risks of each treatment?         1.97
12  Does it describe what would happen if no treatment    1
    is used?
13  Does it describe how the treatment choices affect     2
    overall quality of life?
14  Is it clear that there may be more than one possible  1.78
    treatment choice?
15 Does it provide support for shared decision making?    1.21
16 Based on the answers to all of the above questions,    1.95
   rate the overall quality of the publication as a
   source of information about treatment choices.

Table 4. JAMA benchmarks and percentages

                     Percentage of Websites
JAMA Benchmarks  n   Adhering to Principle

Authorship       28         48%
Attribution       4          6.8%
Disclosure       43         74%
Currency          4          6.8%
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2018 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Olkun, Hatice Kubra; Demirkaya, Arzu Ari
Publication:Turkish Journal of Orthodontics
Article Type:Report
Date:Jun 1, 2018
Previous Article:Evaluation of the Compliance of Orthodontists to Infection Control Procedures in Turkey.
Next Article:Cone Beam Computed Tomography in Orthodontics.

Terms of use | Privacy policy | Copyright © 2021 Farlex, Inc. | Feedback | For webmasters |