A Best-Worst Scaling based Evaluation for Indian News Websites
ANUPRIYA KAUR
Department of Humanities and Social Sciences,
Jaypee University of Information Technology,
Waknaghat,
INDIA
SAKSHI VARSHNEY
Department of Humanities and Social Sciences,
Jaypee Institute of Information Technology,
Noida,
INDIA
Abstract: - Developments in the media industry in recent years compel media houses to improve their website
quality with the goal of attaining a sustainable competitive advantage through online presence. Here, the author
evaluates the website quality of leading digital news alternatives with the use of Best Worst Method. WebQual,
a widely acknowledged model, has been used to identify the contextually relevant criteria for such a study.
Additionally, using three news websites (NDTV, India Today and Times of India) an attempt has been made to
illustrate the criteria based approach to discern key insights which can aid better designed websites.
Key-Words: - News Websites, MCDM, BWM, WebQual.
Received: March 22, 2022. Revised: October 19, 2022. Accepted: November 23, 2022. Published: December 14, 2022.
1 Introduction
Every sphere of our lives has been affected by the
rapid expansion of internet and ICT technologies
and it led to growth of online media. In
contemporary times it is evident that journalism has
quickly migrated to focus on online content and
print media like newspapers are fast dwindling, [1].
When compared with print media online newspapers
have several advantages such as – agility to report
current information, quick and wide dissemination
to a large set of audience, retrieval and archive
contents, [2]. The study pursues two key objectives
– 1) to identify the vital website aspects evaluated
and rated by web users to evaluate a particular news
website and 2) to present the competitive advantage
of three leading online news portals (NDTV, India
today and Times of India (TOI) on the key website
attributes using the Best–Worst Method (BWM).
This emerges as a fresh approach as there is a lack
of studies which evaluate news websites alternatives
using the MCDM (multi criteria decision making)
approach. A systematic approach is followed and
the findings are presented in the form of tables and
figures which facilitates ease of understanding.
2 Identification of Criteria and
Alternatives for News Websites
Three alternatives which featured amongst the top
ten news websites were identified as – NDTV, India
Today and Times of India (TOI).All of these are
widely acknowledged as legacy brands in the field
of media and journalism and now have a digital
presence with their website. Therefore analyzing
their web presence, presents for other similar
players a sound base to assess and optimally
position their own websites. In the past WebQual,
[3], framework emerged as an apt tool to assess
several dimensions of website quality and therefore
forms the underpinning for this study. Eight
dimensions as have been briefly discussed in Table
1 are a part of this analysis.
3 Methodology
Based on the process outlined by past researchers,
[14, 15], a schematic representation of the process
of BWM is outlined: (1) Defining the problem; (2)
Respondents express the best and the worst
criterion; (3) Respondents rate their preference of
the best criterion over all other criteria; (4)
Respondents rate their preference of all other
WSEAS TRANSACTIONS on INFORMATION SCIENCE and APPLICATIONS
DOI: 10.37394/23209.2022.19.32
Anupriya Kaur, Sakshi Varshney
E-ISSN: 2224-3402
303
Volume 19, 2022
criteria over the worst criterion; (5) Computation of
optimal weights; and (6) Computing the aggregated
score of alternatives.
3.1 Data Collection
In this study the purposive sampling method was
employed to collect data. The respondent profile
was expected to 1) exhibit a likely disposition to
regularly prefer to browse news online and 2) an
ability and understanding to interpret survey
questions and provide responses on several pair
wise comparisons. As a first step a solicitation letter
was distributed to prospective respondents. The
URL of the online news website was provided. They
were advised to browse the same for next 10 days
and their consent was sought to participate in the
survey. 16 respondents (eight male and eight female
in an age from 24 years to 48 years) participated in
the study. MCDM and BWM are meant to focus on
the “quality of the information, [15]” and thus work
on a smaller set of data. Steps 1 to 5 are replicated
for each respondent. Thus this study which
consisted of sixteen digitally literate respondents
helped generate findings which could be deemed
reliable.
4 Application of the BWM
STEP 1 Defining the problem
The starting point is to understand and include the
criteria which will be factored by web users about
the online news portals. NDTV, India Today and
Times of India were the three alternatives for this
study. The assessment criteria are outlined in Table
1.
STEP 2 Respondents express the best and the
worst criterion
As a second step, the responses were sought on -
“which of the eight criteria is the most important for
browsing news websites?” and “which of the eight
criteria is the least important for browsing news
websites?” It is to be understood that the most
important criterion is deemed as the best and the
least important criterion as the worst. Table 2 and
Table 3 present the responses. Information Fit-to-
Task stood as the most noteworthy criterion.
Findings reveal that respondents rated it best six
times whereas ‘response time’ was rated best four
times and so was ‘trust’.
STEP 3 Respondents rate their preference of the
best criterion over all other criteria
In this step, the respondents compare all other
criteria with the best and provide their rating on a
scale of 1-9 and table 2 presents the results on the
same. This vector is expressed as
AB = (ab1, ab2,......abn) .................................(1)
“Here, abj indicates the preference of the best
criterion B over the criterion j, [14,15,18]” .
STEP 4 Respondents rate their preference of all
other criteria over the worst criterion
Next, table 3 presents the other to the worst vector.
Respondents now provide their ratings on all other
criteria in comparison with the worst criterion on a
scale of 1-9. Equation 2 provides the mathematical
expression on the same.
AW = (a1w, a2w,.............anw) (2)
Here, ajw indicates the preference of the criterion j
over the worst criterion W.
WSEAS TRANSACTIONS on INFORMATION SCIENCE and APPLICATIONS
DOI: 10.37394/23209.2022.19.32
Anupriya Kaur, Sakshi Varshney
E-ISSN: 2224-3402
304
Volume 19, 2022
WSEAS TRANSACTIONS on INFORMATION SCIENCE and APPLICATIONS
DOI: 10.37394/23209.2022.19.32
Anupriya Kaur, Sakshi Varshney
E-ISSN: 2224-3402
305
Volume 19, 2022
STEP 5 Computation of optimal weights
For this step, an attempt is made to minimize the
maximum absolute
differences for
all j to compute the optimal weights of a criterion.
As outlined by Rezaei [14], the minimization
problem is stated as:
......................(3)
for all j
Based on the process outlined by by Rezaei [14]
the equation (3) can be solved as a linear
optimization model as explained below in (4)
min
s.t.
, for all j
for all j (4)
wj≥ 0, for all j
The solution to equation (4) provides us the optimal
weights ) as well as the optimal
value of . is the consistency ratio of the pair-
wise comparison as a part of the process undertaken
by BWM. The optimal weights are presented in
Figure 1. The highest weight is for ‘Information Fit-
to-Task’ closely followed by ‘Trust’ and ‘Response
Time’. Relative Advantage is ranked lowest. up
to 0.459 is acceptable for studies with eight criteria,
[16]. The average consistency ratio is 0.098, and no
individual consistency values violated the maximum
threshold.
STEP 6 Computing the aggregated score of
alternatives
The study particularly seeks to decipher the
competitive positioning of the chosen alternatives
based on the ratings provided by the respondents. In
this step respondents were to rate each alternative on
each criterion on a scale of 1-9 where 1 refers to
‘not competitive at all’ and 9 refers to ‘extremely
competitive’. To make a valid comparison it was
critical to normalize the score and therefore linear
normalized approach is followed. Therefore each
value is divided by their column-wise maximum
value to obtain normalized scores. Then the
multiplication of each of the normalized values by
their respective weights is computed. A row wise
total is thus obtained and the final priority score is
computed as presented in table 5. The mathematical
expression is as follows
Here, Zi is the final priority value of the alternative i
and xijnorm denotes the normalized values of the
criterion j under the alternative i. Overall, NDTV is
the most preferred online news website, followed by
India Today and then Times of India. However the
mean scores do not indicate too much disparity and
it indicates the alternatives are close competitors to
each other. Furthermore, the aggregate priorities, the
average of all respondents is calculated (Table 5).
NDTV achieves the highest priority across all
criteria. Thus, we can reflect upon the other two
alternatives (India today and TOI). With respect to
trust both score equally, for Visual Appeal,
Information to fit task, Tailored information,
Intuitive operations, Relative advantage India today
scored higher than TOI. Figure 2 facilitates this
understanding.
5 Discussion and Conclusion
Results of this study indicate that NDTV emerged as
the post preferred alternative. If we were to reflect
on the individual priorities of the respondents (Table
4) 15 out of 16 had NDTV as priority. However, a
notable aspect is that all three alternatives stand
close to each other NDTV (1.08), India Today
(0.90) and TOI (0.85). In fact there is an
insignificant gap between India Today and TOI. The
answer to the difference in priority can be
contextualized to the criteria ratings and optimal
weights and are discussed ahead. Table 6 and figure
1 reflect the respective scores across the three
alternatives on these criteria. NDTV has a clear
advantage over the other two alternatives for
Information Fit-to-Task,’ Trust and Response Time
while TOI is having a better score for Response
WSEAS TRANSACTIONS on INFORMATION SCIENCE and APPLICATIONS
DOI: 10.37394/23209.2022.19.32
Anupriya Kaur, Sakshi Varshney
E-ISSN: 2224-3402
306
Volume 19, 2022
Time than India Today. For web managers of these
and similar news websites the implications are clear.
It is evident that web users do not want mental strain
and in fact most of them want to balance their time
and effort to seek information on current affairs and
other significant interests in the form of well
structured news items which are accurate and
timely. Further, to gain credibility news content
should not have a partisan flavor which builds trust
and credibility. Various web optimization tools may
be adopted to improve on the response time and
short news items on news in brief kind of options
may be introduced for users who want to economize
on time. All three alternatives have similar ratings
for intuitive operations. With respect to tailored
information there is a clear difference between the
alternatives with NDTV receiving the highest
followed by India today and TOI. In case of visual
appeal NDTV and India today have a similar rating
while it is noticeably less of TOI. There are some
universal factors which add to visual appeal such as-
a clear uncluttered look with an evident presence of
blank white space, balanced use of colour and
patterns, right font choice and less of advertised
content or sponsored content which inhibit
readability. News websites users hold less
importance for innovativeness and relative
advantage which are perhaps less factored by web
users for news websites in comparison for
merchandise based retailing websites. Nevertheless,
they are not to be neglected and Fig 2 demonstrates
a difference between the alternatives on the same. In
the contemporary tech environment there are
several tools available for website managers which
provide analysis not only for user profile etc for
their site but also for competing sites for example
Google analytics. “Google Analytics is the web
analytics service offered by Google to track and
report the website traffic, [19]”. Site administrators
who are authorized to use Google analytics to assess
their site performance benefit from metrics such as –
number and count of visitation, geographical
location of users, most visited pages and articles etc.
Web managers and site managers may design
strategies around the key criteria such as –
Information Fit-to-Task, Trust, Response time and
Visual appeal and monitor the response using web
analytical tools in a periodic and timely manner.
This would help to optimally position the news
website and augment its competitiveness.
India is the second largest (China being first) online
news consuming nation in the world, [17]. There is
a growing presence of news websites both from
legacy brands and new players making this Industry
fiercely competitive. Website administrators remain
concerned about the strengths and weaknesses of
their website and they would indeed benefit from a
criteria based approach employed by the best worst
method. Benefitting from the direction provided
from this study, strategy development could lead to
increase the competitiveness of the site and pave the
way for monetization [20,21]. Respondents in this
study were limited to 16 and were evident users of
computers and therefore in future studies may be
carried across a diverse set of consumers and
possible inclusion of more criteria.
6 References
[1] Rogers, T. (2019). Are Newspapers Dead or
Adapting in the Age of Digital News? [online]
Available at:
https://www.thoughtco.com/adapting-in-the-
age-of-digital-news-consumption-2074132
[Accessed 29 Mar. 2022].
[2] Haneefa, K.M. and Nellikka, S. (2010),
“Content analysis of online English
newspapers in India”, DESIDOC Journal of
Library & Information Technology, Vol. 30
No. 4, pp. 17-24.
[3] Loiacono, E.T., Watson, R.T. and Goodhue,
D.L. (2007), “WebQual: an instrument for
consumer evaluation of web sites”,
International Journal of Electronic Commerce,
Vol. 11 No. 3, pp. 51-87
[4] McKnight, D.H., Choudhury, V. and Kacmar,
C. (2002), “Developing and validating trust
measures for e-commerce: an integrative
typology”, Information Systems Research,
Vol. 13 No. 3, pp. 334-359.
[5] Longstreet, P., Brooks, S., Featherman, M.
and Loiacono, E., 2021. Evaluating website
quality: which decision criteria do consumers
use to evaluate website quality?. Information
Technology & People.
[6] Geissler, G., Zinkhan, G. and Watson, R.T.
(2001), “Web home page complexity and
communication effectiveness”, Journal of the
Association for Information Systems, Vol. 2
No. 1, available at:
http://aisel.aisnet.org/jais/vol2/iss1/2/
(accessed 14 March 2009)
[7] Elliott, M.T. and Speck, P.S. (1998),
“Consumer perceptions of advertising clutter
and its impact across various media”, Journal
of Advertising Research, Vol. 38 No. 1, pp.
29-41.
[8] Cappiello, C., Francalanci, C. and Pernici, B.
(2003), “Time-related factors of data quality
in multichannel information systems”, Journal
WSEAS TRANSACTIONS on INFORMATION SCIENCE and APPLICATIONS
DOI: 10.37394/23209.2022.19.32
Anupriya Kaur, Sakshi Varshney
E-ISSN: 2224-3402
307
Volume 19, 2022
of Management Information Systems, Vol. 20
No. 3, pp. 71-91.
[9] Rust, R.T. and Lemon, K.N., 2001. E-service
and the consumer. International journal of
electronic commerce, 5(3), pp.85-101.
[10] Rose, G.M. and Straub, D.W. (2001), “The
effect of download time on consumer attitude
toward the e-service retailer”, e-Service
Journal, Vol. 1 No. 1, p. 55.
[11] Galletta, D.F., Henry, R., McCoy, S. and
Polak, P. (2004), “Web site delays: how
tolerant are users?”, Journal of the
Association for Information Systems, Vol. 5
No. 1, pp. 1-28.
[12] Nielsen, J., 2004. Change the Color of Visited
Links, available at:
http://www.useit.com/alertbox/20040503.html
(accessed 29 March 2022).
[13] Laudon, K. and Traver, C. (2017), “e-
Commerce 2017: business, technology,
society”, Pearson, available at:
https://books.google.com/books?id57tIVMQA
ACAAJ.
[14] Rezaei J. (2016) Best-worst multi-criteria
decision-making method: some properties and
a linear model. Omega 64:126–130.
[15] Munim, Z.H., Saha, R., Schøyen, H., Ng,
A.K. and Notteboom, T.E., 2021.
Autonomous ships for container shipping in
the Arctic routes. Journal of Marine Science
and Technology, pp.1-15.
[16] Liang F, Brunelli M, Rezaei J (2019)
Consistency issues in the best worst method:
measurements and thresholds. Omega ,96,
p.102175.
[17] Singh, N. (2019). Trends in Online News
Consumption in India. [online] Available at:
https://www.comscore.com/Insights/Presentati
ons-and-Whitepapers/2019/Trends-in-Online-
News-Consumption-in-India [Accessed 29
Mar. 2022].
[18] Kaur, A. (2022). Empirical Evaluation of
Indian Tourism Websites, 22(9), 63–68.
[19] Kapoor, L. (2021). Benefits of
GoogleAnalytics.[online] Available at
https://www.linkedin.com/pulse/benefits-
google-
analytics-lavanya-kapoor/ [Accessed 29 Mar.
2022].
[20] Kateryna K, Nataliia K., Oleksandr M. V. R.,
"Digitalization of Business Processes of
Enterprises of the Ecosystem of Industry 4.0:
Virtual-Real Aspect of Economic Growth
Reserves," WSEAS Transactions on Business
and Economics, vol. 18, pp. 569-580, 2021.
[21] Stavros K., "Business Trust and
communication as Predictors of Business
Growth," WSEAS Transactions on Business
and Economics, vol. 18, pp. 1512-1522, 2021.
Creative Commons Attribution License 4.0
(Attribution 4.0 International, CC BY 4.0)
This article is published under the terms of the
Creative Commons Attribution License 4.0
https://creativecommons.org/licenses/by/4.0/deed.en
_US
WSEAS TRANSACTIONS on INFORMATION SCIENCE and APPLICATIONS
DOI: 10.37394/23209.2022.19.32
Anupriya Kaur, Sakshi Varshney
E-ISSN: 2224-3402
308
Volume 19, 2022
Table 2 : Best to others vector
Respondent
Best
TR
VA
IN
IFT
TI
RT
IO
RA
1
IFT
3
4
6
1
2
2
3
5
2
TI
2
4
4
2
1
4
4
5
3
TR
1
5
4
2
2
3
3
4
4
TR
1
3
3
2
3
2
2
4
5
IFT
2
4
6
1
3
2
3
6
6
IFT
2
2
3
1
3
2
2
6
7
RT
2
4
4
2
5
1
2
5
8
IO
2
2
3
3
3
2
1
4
9
RT
2
3
2
3
3
1
3
3
10
IFT
2
5
4
1
2
2
3
4
11
TR
1
3
3
3
3
2
2
3
12
RT
2
3
3
2
2
1
3
4
13
IFT
2
4
5
1
2
2
2
4
14
TR
1
2
2
2
2
2
3
3
15
RT
2
3
3
2
3
1
3
4
16
IFT
3
3
6
1
2
3
3
4
1 indicates equal importance and 9 indicates absolutely more important
TR = Trust,VA = Visual Appeal,IN= Innovativeness,IFT = Information Fit-to-Task,TI =Tailored Information,RT = Response
Time,IO = Intuitive Operations,RA = Relative advantage
Table 3 : Others to worst vector
Respondent
Best
TR
VA
IN
IFT
TI
RT
IO
RA
1
IN
3
4
1
4
2
2
3
5
2
RA
6
4
5
5
6
5
5
1
3
IN
7
4
1
6
6
6
6
3
4
RA
7
4
3
5
5
5
5
1
5
VA
4
1
2
6
5
2
3
4
6
IN
5
4
1
6
4
4
4
4
7
T1
6
3
5
5
1
7
4
4
8
IN
5
3
1
5
5
3
3
4
9
RA
6
3
5
6
6
5
5
1
10
RA
4
3
3
6
5
2
3
1
11
RA
6
4
3
5
5
5
5
1
12
VA
7
1
2
6
5
2
3
4
13
RA
4
3
4
6
5
1
3
4
14
TI
7
3
5
5
1
7
4
4
15
IN
5
3
1
5
5
7
3
4
16
VA
5
1
2
7
6
2
3
4
1 indicates equal importance and 9 indicates absolutely more important
TR = Trust,VA = Visual Appeal,IN= Innovativeness,IFT = Information Fit-to-Task,TI =Tailored Information,RT = Response
Time,IO = Intuitive Operations,RA = Relative advantage
WSEAS TRANSACTIONS on INFORMATION SCIENCE and APPLICATIONS
DOI: 10.37394/23209.2022.19.32
Anupriya Kaur, Sakshi Varshney
E-ISSN: 2224-3402
309
Volume 19, 2022
0.000
0.020
0.040
0.060
0.080
0.100
0.120
0.140
0.160
0.180
TR
VA
IN
IFT
TI
RT
IO
RA
Fig 2: Aggregation of criteria value for each alternative
NDTV
India Today
TOI
WSEAS TRANSACTIONS on INFORMATION SCIENCE and APPLICATIONS
DOI: 10.37394/23209.2022.19.32
Anupriya Kaur, Sakshi Varshney
E-ISSN: 2224-3402
310
Volume 19, 2022
