Measuring Agile Project Management Effectiveness with the Application
of Customer Characteristics, Organizational Influence and Project
Management Methodologies in Indonesia
BRYAN OSVALDO1, AMI FITRI UTAMI1, MOHAMMAD ICHSAN2, SHALIGRAM POKHAREL3
1International Business Program, BINUS Business School International Undergraduate Program,
Bina Nusantara University,
Jakarta,
INDONESIA
2Digital Business Program, BINUS Business School International Undergraduate Program,
Bina Nusantara University, Jakarta,
INDONESIA
3Department of Mechanical and Industrial Engineering, College of Engineering,
Qatar University,
Doha,
QATAR
Abstract: - This paper focuses on quantitative research with constructs such as customer characteristics,
organizational influence, and project management methodologies, and how the constructs affect their relationship
with the dependent variable of agile project management effectiveness. Additionally, partial least squares structural
equation modeling or PLS-SEM is employed for this study as the tool to analyze data. Furthermore, 156 data
samples were gathered for this study which mainly aims for APM practitioners in Indonesia. The findings of this
study indicate that customer characteristics do not support APM effectiveness and the moderating variable of PM
methodology between the relationship of customer characteristics and APM effectiveness is not supported.
However, the results of this study can conclude that customer characteristics positively affect organizational
influence. While organizational influence does positively support APM effectiveness. To improve research on APM
effectiveness in Indonesia, this study contributes to laying the preliminary work for future research.
Key-Words: - Agile Project Management, Customer Characteristics, Organizational Influence, PM methodology,
Effectiveness, PLS-SEM.
Received: April 29, 2024. Revised: November 2, 2024. Accepted: December 3, 2024. Published: December 31, 2024.
1 Introduction
Within the 21st century, an approach known as agile
project management (APM) has been created to
further advance the frameworks of traditional project
management. Although APM is a newer version of
the traditional PM and was developed only 20 years
ago [1], it has also spread versatility to other
industries such as construction and food [2] and
the healthcare industry, [3]. In a KPMG report, not a
single respondent from Brazil has not applied agile
methodologies, and over 40% wish to be agile within
a venture level. Respondents from the Netherlands
indicate that organizations are not contemplating
whether APM should be implemented, instead, they
are figuring out how would they implement it. On the
contrary, respondents from Germany specify that
although APM is already known, waterfall
methodologies are used more frequently, [4]. The
results of the 2021 KPMG and AIPM project
management survey mention that 71% of individual
respondents and 68% of organizations within
Australia have successfully implemented APM either
fully adopting APM or a mix between APM and
traditional PM. Furthermore, the survey informs its
WSEAS TRANSACTIONS on SYSTEMS
DOI: 10.37394/23202.2024.23.44
Bryan Osvaldo, Ami Fitri Utami,
Mohammad Ichsan, Shaligram Pokharel
E-ISSN: 2224-2678
419
Volume 23, 2024
readers that 52% of individual respondents believe
that APM improves success rates among projects,
while only 37% of organizational respondents think
that APM boosts their success rates, thus creating
mixed responses for APM practitioners in Australia,
[5].
Evidence of APM and its benefits have been
researched thoroughly on a global scale. However, to
understand the effectiveness as well as challenges to
its implementation would have to be further
discussed and explored. APM has been researched
globally, however, the presence of research studies
regarding APM in Indonesia is limited, therefore
more reasons to create empirical evidence of APM in
Indonesia. Research conducted by PWC Indonesia
found that the adoption of APM within Indonesian
banking firms is still in the early stages, as only 24%
of Indonesian banks have adopted APM on more than
50% of their projects but not all. However, 76% of
the respondents from Indonesian banking firms
believed that APM would be implemented in
Indonesia in the coming years, [6]. From the two
articles of KPMG Australia and PWC Indonesia, it is
arguable that APM from both countries have been
incorporated into their structure but the benefits from
APM itself have not been thoroughly analyzed. A few
studies on the implementation of APM within
Indonesia imply that the biggest challenge of APM in
Indonesia would be communication variables, [7].
Teamwork quality is also a major component within
APM, especially within startups in Indonesia, [8].
Despite the study [9] that considers factors
including PM methodology, organizational
challenges, and customer-related challenges as
challenges in implementing APM within Indonesian
companies, while waterfall methodology is the
biggest challenge. There is still a lack of evidence on
the significance of the study within Indonesia. This
study provides theoretical frameworks to develop the
hypotheses. The analysis, results, implication of
results, and limitations are also discussed. What
differentiates this paper from other related technical
literature papers is that the effectiveness of APM is
researched within the areas of customers,
organizations, and PM methodology. The current
literature mainly focuses on the implementation of
APM and not the effectiveness of it.
Additionally, this paper aims to measure the
effectiveness of APM in correlation to variables such
as Customer Characteristics, Organizational
Influence, and Project Management methodologies
within Indonesia.
2 Literature Review
2.1 Project Management
Companies that do not transition from traditional
project management (PM) into agile are due to
organizational influence, these companies do not find
fault within traditional PM from previous
experiences, therefore it would be rational for them
to continue implementing waterfall methodologies.
[10], based on empirical research, a company culture
that leans towards hierarchy tends to utilize more of
a waterfall methodology instead of APM, [11].
However, practitioners of traditional PM tend to
experience more challenges when facing projects
with high uncertainty, [12]. Additionally,
practitioners of project management have
acknowledged that traditional project management
methods might not be ideal for planning and
execution as they look for other alternatives such as
APM, [13].
2.2 Agile Project Management
One of the differences between APM and traditional
PM is that APM is capable of adapting to
uncertainties and changes throughout the process of
the project, [14].
Moreover, APM encourages uncertainties and
change to make the most out of the competitive
advantage of the customer, [15]. Another factor
within APM is that there is a high emphasis on
communication and collaboration between customers
or clients and the project team, [16], [17].
To explain the phenomena of APM, three agile
theories can be employed to describe them. The
complex adaptive system theory which emphasizes
interactions and feedback can be defined as a system
that undergoes constant change from uncertainties
within its environment and adapts its rules as learning
experiences progress, [18]. The control theory can be
defined as the attempt by management to ensure that
all parties working on projects would have to follow
a strategic strategy before achieving their goals, [19],
[20]. Moreover, the control theory monitors and
evaluates the behaviors and outcomes of participants
which is an important factor in analyzing team
performance in efficiency and effectiveness, [21],
[22]. The coordination theory refers to the
WSEAS TRANSACTIONS on SYSTEMS
DOI: 10.37394/23202.2024.23.44
Bryan Osvaldo, Ami Fitri Utami,
Mohammad Ichsan, Shaligram Pokharel
E-ISSN: 2224-2678
420
Volume 23, 2024
information that coordination is important in
identifying dependencies within an information
system. Moreover, coordination is an important
factor within agile software development as teams
working on projects within an organization require
coordination from one another, [23], [24]. However,
a study also suggests that coordination theory only
focuses on identifying dependencies and is not
suitable for prediction. Furthermore, coordination
research within information systems has found that
coordination is necessary but does not determine
project success, [23].
2.3 PM Methodology
APM has transitioned into various types within the
software development field such as Scrum [25], XP
also known as extreme programming [26] and
Kanban [27], [28].
2.3.1 Scrum
Scrum is an agile software method that prioritizes
working in sprints, which are iterations that break
down complex projects into smaller parts, [29].
Scrum consists of three factors: product backlog,
sprint backlog, and sprint burnout chart, [30].
2.3.2 Extreme Programming (XP)
Extreme Programming also known as XP is another
type of APM method that employs the principles of
Agile within the manifesto. The differences between
XP with other types of agile methodologies mainly
revolve around its incremental planning approach
which changes accordingly as the project moves on
to the later stages, [31].
2.3.3 Kanban
Kanban is yet another type of APM methodology that
is incorporated within the manufacturing industry.
The main system of Kanban revolves around
delivering raw materials to the next stage of
production only when there is a presence of customer
demand, this means that there would be less waste as
over-production is eliminated thus creating a
sustainable approach, [32].
PM methodology can be defined as a manual or
guide for PM practitioners to manage their projects
effectively and lead to project completion, [33]. PM
methodologies vary from a wide range of types such
as Scrum, Extreme Programming, and Kanban.
Furthermore, the choice of PM methodology itself
might affect the organizational influence as well as
the effectiveness of APM. A study regarding project
management methodology usage explains that 35.3%
of their respondents tend to use Scrum while 29.8%
of respondents frequently use waterfall methodology.
It is also stated that PM methodology should be
tailored according to the sector in which the
organisation operates, [34]. The correlation between
PM methodology and organizational influence would
have to be researched further.
From an effectiveness standpoint, choosing a
random PM methodology and following it would not
lead to success and its benefits such as ease of project
control and effectiveness would not be achieved,
[33], [35]. For example, a study conducted on
repetitive construction companies concluded that
the PRINCE2 project methodology is the most
suitable for the organization as its guidelines allow
the company to provide as much information as
possible to team members, [36]. Ultimately, the
effectiveness of PM methodology comes from the
choice of methodology. However, the study of PM
methodology in correlation to APM effectiveness has
not been done within Indonesia.
There are three factors to be considered when
measuring the PM methodology effectiveness. The
first is Development Practice, which addresses best
practices within the Agile technique, specifically
pertaining to the Scrum Framework. Second, product
ownership is a crucial scrum project stakeholder that
influences the project's overall performance by
establishing priorities, setting direction, and ensuring
quality. Thirdly, for long-term success, Teams as
Scrum stresses a self-organizing, cross-functional
team with a committed Product Owner, Scrum
Master, and Development Team all collaborating in
one place, [37]. Additionally, flexibility is also
another important part of PM methodology as it
allows project managers to adapt accordingly to
emergencies within a dynamic environment, [38].
2.3 Customer Characteristics
Customer Characteristics can be defined as the users'
participation in the creation process with lead-user
attributes that increase the possibility of creating
offerings with higher value, [39].
Customer involvement is a dimension within
customer characteristics, this dimension can be
defined as the perception among customers that they
are involved in the business, [40]. On the contrary,
another study indicates that customer involvement
WSEAS TRANSACTIONS on SYSTEMS
DOI: 10.37394/23202.2024.23.44
Bryan Osvaldo, Ami Fitri Utami,
Mohammad Ichsan, Shaligram Pokharel
E-ISSN: 2224-2678
421
Volume 23, 2024
refers to the active participation of customers in the
creation of a product, [41].
To obtain positive relationships with customers
and to understand further their concerns, customer
communication is an important factor. According to a
study, in order to achieve customer satisfaction,
understanding what customers require is essential,
[42]. Customer knowledge also helps in creating an
innovation of a product. A study mentions that
acquiring, interpreting, sharing, and applying
customer insights would potentially improve the total
outcome of a product, [43].
2.4 Organizational Influence
Organizational influence can be defined as a set of
beliefs, norms, values, attitudes, and assumptions
within employees that controls the organization.
Moreover, achieving company tasks and the
behaviour of the workforce are greatly affected by
these elements, [44]. Conclusions from a study point
out that organizational influence heavily determines
the types of PM methodology used by organizations.
According to a study that measures the impact of
organizational culture, this indicator can be described
as the mindset that differentiates one particular group
project from another including personal cultural
differences, [45]. Another study defines
organizational culture as learning feedback from
senior staff which increases performance for an agile
team, [46]. Additionally, a study conducted with a
sample of mobile app companies in Saudi Arabia
suggests that organizational culture directly affects
the effectiveness of APM within the company.
Furthermore, it is also stated that when the
environment does not allow employees to have the
freedom to express opinions and ideas, there is a
chance that APM will not be able to adapt
successfully, [47].
Organizational structure that follows hierarchical
cultures tends to be formal and following a structure
is necessary which promotes stability. Furthermore,
planning and low costs define success within a
hierarchical culture, [11]. Within a hierarchical
culture, top management will implement written rules
and responsibilities over the lower-level
management. Moreover, it is also known that
organizational members will be informed of the
process for group activities, [48]. On the contrary,
another type of organizational structure is known as
organic organization structure. Furthermore, this type
of structure emphasizes flatness within the whole
structure. Additionally, communication and sharing
of ideas regarding the process and other product-
related ideas between lower-level and top
management are deeply encouraged, [48], [49].
Monitoring and controlling within the
organizational dimension is regarded as the ability to
monitor and control individuals within teams to
create project success, [50]. Moreover, the span of
control is also synonymous with this indicator, and
the latter is defined as the amount of junior staff that
can be successfully guided by a supervisor, [51]. The
findings within a study between monitoring and
control with project management report that
monitoring and control do have a positive impact on
the project performance within the scope of time,
cost, quality, and customer satisfaction by
approximately 22%. In addition, this is also vice
versa as when the value of monitoring and
controlling is lower, so does the project performance,
[52]. However, it should be noted that the findings
were based on one Indonesian company, and thus the
results could not be accurately true for other
companies.
2.5 APM Effectiveness
Metrics can be used to measure the effectiveness of
APM within a particular project. Burn-down rate is a
metric that measures the remaining work within a
sprint, which enables the predictability and progress
monitoring for the project. Furthermore, when
the burn-down rate decreases, this means that
management has successfully reduced workload and
this leads to APM effectiveness, [53]. Additionally,
team velocity is also a type of metric used to measure
APM effectiveness. Team velocity can be defined as
the speed at which work is done by the project team
and most practitioners of Scrum tend to use this
metric to understand APM effectiveness. However, a
study argued that using team velocity might lead to
negative effects as it would make different teams
uncomfortable as starting points are different than
one another in Scrum, [54]. Although it is also
important to note that the study was measured within
the PM methodology of Scrum, other types of PM
methodology might benefit from executing team
velocity.
While lead time refers to the amount of time
spent in each stage for each requirement or user story,
defect state overtime refers to the rate at which
defects are introduced, the rate at which defects are
analyzed, designed, and implemented, and the rate at
WSEAS TRANSACTIONS on SYSTEMS
DOI: 10.37394/23202.2024.23.44
Bryan Osvaldo, Ami Fitri Utami,
Mohammad Ichsan, Shaligram Pokharel
E-ISSN: 2224-2678
422
Volume 23, 2024
which corrections packages solutions are
implemented for deployment at customer sites, [55].
A metric called customer satisfaction assesses how
happy customers are with the final product, [56].
Whereas "quality of the result" is the difference
between the quality of the request and the final
goods, "delivered business results" refers to the
promptness and accuracy of the sought result, [57].
Figure 1 explains the Research Model which includes
the 4 hypotheses of this particular study, [57].
Fig. 1: Research Model
H1: Customer Characteristics positively affects APM
effectiveness significantly.
H2: Customer Characteristics positively affects
Organizational Influence significantly.
H3: Organizational Influence positively affects APM
effectiveness significantly.
H4: PM methodology significantly moderates
positively the relationship between Customer
Characteristics and APM effectiveness.
3 Research Methodology
3.1 Sample and Procedure
This study will be categorized as quantitative
research which plans to identify the impact of PM
methodology, organizational influences, and team
challenges in relation to APM effectiveness. In this
paper, a mixed-methods approach is utilized through
questionnaires, surveys, statistical analysis, and a
quantitative framework. Moreover, the collection of
data will be conducted by a reputable and trusted
third party and the data analysis will consist of
Indonesian APM practitioners. A cross-sectional time
horizon is also employed which obtains data at a
longer period. In addition, this quantitative research
will use convenience sampling, as the trusted third
party provider will be choosing the participants that
have met the criteria and are available. Finally,
Structural Equation Modelling with Partial Least
Squares (SEM-PLS) with a chosen software of Smart
PLS 4.0 will be implemented. The Smart PLS 4.0 is
known to identify complex correlations and sampling
biases.
3.2 Data Measurement
A 6-point Likert Scale (1=strongly disagree,
6=strongly agree) would be used to measure data
obtained from Indonesian APM practitioners as
recommended by a study when compared to
alternative scales, [58]. This increases the integrity,
validity as well as the quality of data collected from
participants.
3.3 Data Analysis
Data will be analyzed using Smart PLS 4.0 software,
as this allows the evaluation of measurement and
structural models. Additionally, convergent and
discriminant validation was used in the measurement
model assessment to validate the applicability and
reliability of the indicators. Average variance
extracted (AVE) more than 0.5 and outer loadings
greater than 0.7 were used to establish convergent
validity. Heterotrait-Monotrait (HTMT) ratios were
used to assess discriminant validity; for each
indicator, the cross-loadings should be smaller than
the outer loading for the target construct. The average
inter-construct correlation divided by the average
intra-construct correlation is known as the HTMT
criterion, and it should range between 0.85 to 0.90.
Path coefficient significance, predictive
relevance (Q2), and coefficient of determination (R2)
were used to assess the structural model. Variance
inflation factors (VIF) smaller than five were used to
measure the degree of collinearity. It is also
recommended to implement bootstrapping of 5000
subsamples in order to dictate the importance of path
coefficients, [59]. Additionally, the effect size (f2)
was employed to calculate how missing components
affected endogenous variables, [60]. Model fit was
evaluated using an RMSEA smaller than 0.08 and a
WSEAS TRANSACTIONS on SYSTEMS
DOI: 10.37394/23202.2024.23.44
Bryan Osvaldo, Ami Fitri Utami,
Mohammad Ichsan, Shaligram Pokharel
E-ISSN: 2224-2678
423
Volume 23, 2024
standardized root mean square residual (SRMR)
because PLS-SEM does not imply normality.
4 Results and Discussion
A sum of 300 samples was requested under a
reputable third-party data collection service provider.
Additionally, a convenience sampling method was
utilized in order to select the sample from the
population. After careful consideration through
filtering eligibility from respondents, 156 data
samples were deemed to be valid for further research.
Data was analyzed using the partial least squares
structural equation model (PLS-SEM) with support
from SmartPLS version 4.0. Furthermore, in order to
determine that the data collected were reliable and
valid, three measurement model analyses known as
the Heterotrait-Monotrait Ratio of the correlations
(HTMT), the average variance extracted (AVE), and
Cronbrach’s alpha/composite reliability (CR) were
utilised for further study. A two-step analysis
approach was also employed for effective research,
[60].
The first step in the two-step analysis is to
conduct a descriptive analysis which contains the
demographics of the sample, this is shown in Table 1
(Appendix). The second step is to create a structural
model analysis which consists of the measurement
model analysis, structural model analysis, and
explanatory model analysis to identify the Variance
Inflation Factor (VIF) utilized for collinearity.
Based on the data presented, most samples were
females and respondents mostly had a bachelor’s
degree. Most of the respondents tend to have 1-5
years of experience working within the fields of APM
and most were Business Analysts. Furthermore,
respondents mostly come from the industrial sector.
4.1 Measurement Model Assessment
To test the reliability of the data, the items of the
construct are required to be greater than 0.7 in terms
of Cronbach’s Alpha and factor loadings. Moreover,
loading values of less than 0.7 should be avoided for
data to be reliable, [59]. Additionally, Composite
Reliability measures the dependability of data, and
values exceeding 0.7 indicate that the construct tends
to have more dependability. When values of the
Average Variance Extracted are more than 0.5, this
means that the construct accounts for more than half
of the difference between its indicators. Table 2
(Appendix) indicates the measurement model
analysis result which indicates the Cronbach’s
Alpha, factor loadings, composite reliability, and
average variance extracted for this study.
Values in Table 3 (Appendix) indicate the HTMT
ratio of the study. Values below 0.9 would mean that
indicators are not highly correlated to one another.
Thus, proving that discriminant validity has been
accomplished between two indicators. The
standardized root mean square residual of this model
was 0.069 which fits the suggested model criteria of
0.08, [59].
To further explain Table 4 (Appendix), the R2 or
value of the coefficient of determination for APM
effectiveness is 0.663. This means that 66.3% of
Customer Characteristics, Organizational Influence,
and PM methodology are associated with APM
effectiveness, whereas 33.7% of the variables are
assigned to other factors outside the model.
Additionally, the R2 for Organizational Influence is
0.408 which means that 40.8% of Customer
Characteristics accounts for the variables related to
Organizational Influences. The Q2 or predictive
relevance for APM effectiveness is 0.381 and 0.373
for Organizational Influences, this means that lower
levels of accuracy exist within the PLS predictive
path model. However, the values of Q2 are acceptable
as they are more than zero.
The F2 is used to indicate how much an
independent variable influences the dependent
variable, Table 5 (Appendix) shows the effect size
between an independent variable towards the
dependent variable which is APM effectiveness. In
this case, it is safe to assume that out of all the
independent variables of the model, Organizational
Influence tends to have the highest effect size APM
effectiveness with a value of 0.6.
The PLS predict function was utilized in order to
identify whether the model contains predictive
power. The results in Table 6 (Appendix) show that
only two items contain a higher RMSE PLS than
RMSE LM. This means that only 2 out of 14 items
possess bigger predictive errors. The table shows that
the model contains better prediction accuracy as
stated by the PLS predict function.
4.2 Measurement Model Assessment
The Variance Inflation Factor (VIF) is used to
indicate the existence of collinearity between
variables. If the value of VIF is between 3 to 5 this
would mean that collinearity exists, [59]. Table 7
(Appendix) shows that all values were less than 3
WSEAS TRANSACTIONS on SYSTEMS
DOI: 10.37394/23202.2024.23.44
Bryan Osvaldo, Ami Fitri Utami,
Mohammad Ichsan, Shaligram Pokharel
E-ISSN: 2224-2678
424
Volume 23, 2024
which would mean that the constructs were not
collinear to one another.
Bootstrapping of 5000 sub-samples was also
performed in order to identify the relevance and
significance of the path coefficient within the model.
Figure 2 (Appendix) shows the results of the
bootstrapping with 5000 sub-samples.
The hypotheses test result in Table 8 (Appendix)
indicates that both H2 and H3 contain a p-value of
less than 0.005 which means that these hypotheses
are supported by the research. While H1 and H4
contain a p-value of more than 0.005, thus making it
not supported by data. It is safe to conclude that
customer characteristics positively affect
organizational influences and organizational
influence positively affects APM effectiveness as the
P value of each of the hypotheses is 0.001.
5 Conclusions
The result of this study, which analysed the responses
from 156 participants with prior experience with
APM usage, determines that customer characteristics
support positively the organizational influence as the
P value of each of the hypotheses is 0.001.
Additionally, organizational influence positively
affects the APM effectiveness which is the dependent
variable in this study as the P value of each of the
hypotheses is 0.001. Additionally, dimensions within
customer characteristics such as customer
involvement, customer satisfaction, customer
communication, and customer knowledge should be
focused upon to increase the construct of
organizational influence. Moreover, organizational
influence which contains dimensions such as culture,
monitoring & control, and organizational structure is
important for APM practitioners in Indonesia to take
into consideration to increase the effectiveness of
APM. Besides, this study also found that customer
characteristics do not have a correlation with the
effectiveness of APM as the P value for this is shown
to be 0.626 and therefore higher than 0.05.
Furthermore, PM methodology as a moderating
variable between customer characteristics towards
APM effectiveness does not support positively as it
contains a P value of 0.681, which is higher than
0.05. However, it is important to note that the
dimensions within PM methodology only contain
sub-variables such as flexibility, product ownership,
development practice, and teams. It might be
beneficial for future research to conduct more
exploration on other dimensions within PM
methodology such as project complexity and risk
tolerance to name a few.
This study discusses the hypotheses regarding
APM in Indonesia. Commercial businesses that
utilize APM to increase APM efficiency. The findings
of this research are encouraged to be used as
recommendations for further studies in other
commercial industries in Indonesia. More research is
also encouraged on the dimensions of organizational
influence and customer characteristics. However,
this study does not include the specificities of each
construct such as independent traits regarding
customer characteristics, organizational influences,
and PM methodology. Furthermore, this will provide
additional room for research for future studies. In
order to improve the studies on APM effectiveness,
this research will purposely lay future groundwork
for further studies on APM in Indonesia.
Future research on the effectiveness of APM
could be done on other variables aside from customer
characteristics, organizational influence, and PM
methodologies. Additionally, variables such as the
emerging trend of Artificial Intelligence as well as
digital leadership might play a huge role in
determining the effectiveness of APM.
References:
[1] F. P. Zasa, A. Patrucco, and E.
Pellizzoni, “Managing the Hybrid
Organization: How Can Agile and
Traditional Project Management
Coexist?,” Research Technology
Management, vol. 64, no. 1, 2020, doi:
10.1080/08956308.2021.1843331.
[2] V. Goldstein and J. Euchner,
“Transformation for Growth at GE,”
Research-Technology Management, vol.
60, no. 6, pp. 14–19, Nov. 2017, doi:
10.1080/08956308.2017.1373045.
[3] V. Roblek, V. Dimovski, K. Jovanov
Oblak, M. Meško, and J. Peterlin,
“Leadership and managerial challenges
to ensure agile management as a
method to enable business success: a
Delphi study of the Slovenian health
organisations,” Measuring Business
Excellence, Vol. 28 No. 1, pp. 39-51,
Nov. 2023, doi: 10.1108/MBE-09-2023-
0122.
WSEAS TRANSACTIONS on SYSTEMS
DOI: 10.37394/23202.2024.23.44
Bryan Osvaldo, Ami Fitri Utami,
Mohammad Ichsan, Shaligram Pokharel
E-ISSN: 2224-2678
425
Volume 23, 2024
[4] KPMG, “2019 Survey on Agility Agile
Transformation From Agile experiments
to operating model transformation:
How do you compare to others?,” 2019,
[Online].
https://assets.kpmg.com/content/dam/kp
mg/pe/pdf/Publicaciones/TL/agile-
transformation.pdf (Accessed Date:
November 3, 2024).
[5] KPMG, “2021 AIPM and KPMG
Project Management Survey,” 2021,
[Online]. https://pmworldlibrary.net/wp-
content/uploads/2021/11/211104-2021-
AIPM-and-KPMG-Project-
Management-Survey-Report.pdf
(Accessed Date: November 3, 2024).
[6] PWC, “Digital Banking in Indonesia
2018”, [Online].
https://www.pwc.com/id/en/publication
s/assets/financialservices/digital-
banking-survey-2018-pwcid.pdf
(Accessed Date: November 3, 2024).
[7] S. Sujono, M. A. Setiawan, and K.
Haryono, “Tantangan Adopsi Agile di
Perguruan Tinggi di Indonesia,” JUITA:
Jurnal Informatika, vol. 8, no. 2, p. 197,
Nov. 2020, doi:
10.30595/juita.v8i2.7217.
[8] M. Fathin and T. Raharjo, “Critical
Factors to Improve Teamwork Quality
in Indonesian Startups Using aTWQ
Framework,” Jurnal RESTI (Rekayasa
Sistem dan Teknologi Informasi), vol. 7,
no. 5, pp. 1246–1252, Oct. 2023, doi:
10.29207/resti.v7i5.5099.
[9] S. , A. Rahmah, N. , R. Pratama, S. , A.
Kuswadi, and M. Ichsan, “The
Effectiveness of Implementing Agile
Project Management: A Systematic
Literature Review,” Global Business
and Finance Review, Vol.29, No.6, p.
170-186, 2023,
https://doi.org/10.17549/gbfr.2024.29.6.
170.
[10] F. Albuquerque, A. S. Torres, and F. T.
Berssaneti, “Lean product development
and agile project management in the
construction industry,” Revista de
Gestão, vol. 27, no. 2, pp. 135–151,
Apr. 2020, doi: 10.1108/REGE-01-
2019-0021.
[11] K. Piwowar-Sulej, “Organizational
culture and project management
methodology: research in the financial
industry,” International Journal of
Managing Projects in Business, vol. 14,
no. 6, pp. 1270–1289, Aug. 2021, doi:
10.1108/IJMPB-08-2020-0252.
[12] M. Bianchi, G. Marzi, and M. Guerini,
“Agile, Stage-Gate and their
combination: Exploring how they relate
to performance in software
development,” J Bus Res, vol. 110, pp.
538–553, Mar. 2020, doi:
10.1016/j.jbusres.2018.05.003.
[13] P. Serrador and J. K. Pinto, “Does Agile
work? — A quantitative analysis of
agile project success,” International
Journal of Project Management, vol.
33, no. 5, pp. 1040–1051, Jul. 2015,
doi: 10.1016/j.ijproman.2015.01.006.
[14] T. Cooke-Davies, “‘The “real” success
factors on projects,’” International
Journal of Project Management, vol.
20, no. 3, pp. 185–190, 2002,
http://dx.doi.org/10.1016/S0263-
7863(01)00067-9.
[15] H. Salameh, “What, When, Why, and
How? A Comparison between Agile
Project Management and Traditional
Project Management Methods,”
International Journal of Business and
Management Review, vol. 2, no. 5, pp.
52–74, Oct. 2014, [Online].
https://eajournals.org/ijbmr/vol-
2issue5october-2014/comparison-agile-
project-management-traditional-project-
management-methods/ (Accessed Date:
November 26, 2024).
[16] S. Collyer, C. Warren, B. Hemsley, and
C. Stevens, “‘Aim, fire, aim project
planning styles in dynamic
environments,’” Project Management
Journal, vol. 41, no. 4, pp. 108–121,
2010,
http://dx.doi.org/10.1002/pmj.20199.
[17] K. Aguanno, Managing Agile Projects,
vol. 35. Lakefield: Multimedia
Publications, 2004, [Online].
https://www.researchgate.net/publicatio
n/234797070_Managing_Agile_Project
s (Accessed Date: November 26, 2024).
WSEAS TRANSACTIONS on SYSTEMS
DOI: 10.37394/23202.2024.23.44
Bryan Osvaldo, Ami Fitri Utami,
Mohammad Ichsan, Shaligram Pokharel
E-ISSN: 2224-2678
426
Volume 23, 2024
[18] G. Alaa and G. Fitzgerald,
“Conceptualizing Service-Based
Information System Evolution as a
Complex Adaptive System ,”
Emergence: Complexity and
Organization, vol. 15, no. 3, pp. 1–23,
2013, [Online].
https://www.researchgate.net/publicatio
n/266410865_Conceptualizing_Service-
Based_Information_System_Evolution_
as_a_Complex_Adaptive_System
(Accessed Date: November 26, 2024).
[19] L. M. Maruping, V. Venkatesh, and R.
Agarwal, “A Control Theory
Perspective on Agile Methodology Use
and Changing User Requirements,”
Information Systems Research, vol. 20,
no. 3, pp. 377–399, Sep. 2009, doi:
10.1287/isre.1090.0238.
[20] L. J. Kirsch, “The management of
complex tasks in organizations:
Controlling the systems development
process,” Organization Science, vol. 7,
no. 1, pp. 1–21, 1996,
https://doi.org/10.1287/orsc.7.1.1.
[21] S. R. Nidumolu and M. R. Subramani,
“The Matrix of Control: Combining
Process and Structure Approaches to
Managing Software Development,”
Journal of Management Information
Systems, vol. 20, no. 3, pp. 159–196,
Dec. 2003, doi:
10.1080/07421222.2003.11045774.
[22] J. C. Henderson and S. Lee, “Managing
I/S design teams: A control theories
perspective.,” Manage Sci, vol. 38, no.
6, pp. 757–777, 1992,
https://doi.org/10.1287/mnsc.38.6.757.
[23] D. E. Strode, S. L. Huff, B. Hope, and
S. Link, “Coordination in co-located
agile software development projects,”
Journal of Systems and Software, vol.
85, no. 6, pp. 1222–1238, Jun. 2012,
doi: 10.1016/j.jss.2012.02.017.
[24] T. W. Malone and K. Crowston, “The
interdisciplinary study of coordination,”
ACM Comput Surv, vol. 26, no. 1, pp.
87–119, Mar. 1994, doi:
10.1145/174666.174668.
[25] C. Larman and V. Basili, “Iterative and
incremental development: A brief
history.,” IEEE Computer, vol. 36, no.
6, pp. 47–56, 2003,
https://doi.org/10.1109/MC.2003.12043
75.
[26] K. Beck, “Embracing change with
extreme programming,” Computer
(Long Beach Calif), vol. 32, no. 10, pp.
70–77, 1999, doi: 10.1109/2.796139.
[27] H. Alaidaros, M. Omar, and R. Romli,
“The state of the art of agile kanban
method: challenges and opportunities,”
Independent Journal of Management &
Production, vol. 12, no. 8, pp. 2535–
2550, Dec. 2021, doi:
10.14807/ijmp.v12i8.1482.
[28] A. M. M. Hamed and H. Abushama,
“Popular agile approaches in software
development: Review and analysis,” in
2013 International Conference on
Computing, Electrical and Electronic
Engineering (ICCEEE), Khartoum:
IEEE, Aug. 2013, pp. 160–166. doi:
10.1109/ICCEEE.2013.6633925.
[29] B. Boehm, “Get ready for agile
methods, with care,” IEEE Computer,
vol. 35, no. 1, pp. 64–69, 2002,
https://doi.org/10.1109/2.976920.
[30] K. Schwaber and M. Beedle, Agile
software development with SCRUM.
Upper Saddle River, NJ: Prentice Hall,
2002, [Online].
https://www.agileleanhouse.com/lib/lib/
People/KenSchwaber/Agile%20Project
%20Management%20With%20Scrum%
20-www.itworkss.com.pdf (Accessed
Date: November 26, 2024).
[31] K. Beck and C. Andres, Extreme
Programming explained: Embrace
change, 2nd ed. NJ: Addison-Wesley
Professional, 2004, [Online].
https://dl.acm.org/doi/10.5555/318762
(Accessed Date: November 26, 2024).
[32] J. A. Garza-Reyes, V. Kumar, S.
Chaikittisilp, and K. H. Tan, “The effect
of lean methods and tools on the
environmental performance of
manufacturing organisations,” Int J
Prod Econ, vol. 200, pp. 170–180, Jun.
2018, doi: 10.1016/j.ijpe.2018.03.030.
[33] P. Jovanovic and I. Beric, “Analysis of
the Available Project Management
WSEAS TRANSACTIONS on SYSTEMS
DOI: 10.37394/23202.2024.23.44
Bryan Osvaldo, Ami Fitri Utami,
Mohammad Ichsan, Shaligram Pokharel
E-ISSN: 2224-2678
427
Volume 23, 2024
Methodologies,” Management:Journal
of Sustainable Business and
Management Solutions in Emerging
Economies, vol. 23, no. 3, p. 1, Dec.
2018, doi:
10.7595/management.fon.2018.0027.
[34] G. Chandrachooodan, R. Radhika, and
R. R. Palappan, “Role of project
management methodology in challenges
faced in implementation of Information
Systems projects in the new normal
after pandemic,” Int J Health Sci
(Qassim), pp. 6396–6412, Apr. 2022,
doi: 10.53730/ijhs.v6nS1.6356.
[35] H. Kerzner, “Advanced Project
Management: Best Practices on
Implementation,” John Wiley & Sons,
2004, [Online].
https://www.wiley.com/en-
us/Advanced+Project+Management%3
A+Best+Practices+on+Implementation
%2C+2nd+Edition-p-9780471472841
(Accessed Date: November 26, 2024).
[36] A. Simonaitis, M. Daukšys, and J.
Mockienė, “A Comparison of the
Project Management Methodologies
PRINCE2 and PMBOK in Managing
Repetitive Construction Projects,”
Buildings, vol. 13, no. 7, p. 1796, Jul.
2023, doi: 10.3390/buildings13071796.
[37] R. Roshan and A. Santhosh, “Adoption
of agile methodology for improving IT
project performance,” Serbian Journal
of Management, vol. 16, no. 2, pp. 301–
320, 2021, doi: 10.5937/sjm16-26854.
[38] M. Lindvall, V. Basili, B. Boehm, P.
Costa, K. Dangle, F. Shull, R. Tesoriero,
L. Williams, and M. Zelkowitz,
“Empirical Findings in Agile Methods,”
in Extreme Programming and Agile
Methods - XP/Agile Universe 2002, vol.
2418, Chicago: Springer, 2002, pp.
197–207. doi: 10.1007/3-540-45672-
4_19.
[39] P. Carbonell, A. I. Rodriguez‐Escudero,
and D. Pujari, “Performance effects of
involving lead users and close
customers in new service development,”
Journal of Services Marketing, vol. 26,
no. 7, pp. 497–509, Oct. 2012, doi:
10.1108/08876041211266440.
[40] Y. -C. Chang, W.-H. Chiu, J.-H. Wang,
and M.-J. Teng, “Customer involvement
in the new process innovation:
antecedents, mediation and
performance,” European Journal of
Innovation Management, vol. 25, no. 4,
pp. 1115–1141, Jun. 2022, doi:
10.1108/EJIM-09-2019-0268.
[41] S. Najafi-Tavani, G. Zaefarian, M. J.
Robson, P. Naudé, and F. Abbasi,
“When customer involvement
hinders/promotes product innovation
performance: The concurrent effect of
relationship quality and role ambiguity,”
J Bus Res, vol. 145, pp. 130–143, Jun.
2022, doi:
10.1016/j.jbusres.2022.03.001.
[42] H. C. O. Unegbu, D. S. Yawas, and B.
Dan-asabe, “An investigation of the
relationship between project
performance measures and project
management practices of construction
projects for the construction industry in
Nigeria,” Journal of King Saud
University - Engineering Sciences, vol.
34, no. 4, pp. 240–249, May 2022, doi:
10.1016/j.jksues.2020.10.001.
[43] C. Larbig-Wust, “Measuring customer
involvement in new service
developments,” City University of
London, London, 2010, [Online].
https://openaccess.city.ac.uk/id/eprint/1
140/ (Accessed Date: November 3,
2024).
[44] E. H. Schein, “Organizational culture.,”
American Psychologist, vol. 45, no. 2,
pp. 109–119, Feb. 1990, doi:
10.1037/0003-066X.45.2.109.
[45] M. Muneer, N. Khan, M. Awais
Hussain, Z. Shuai, A. A. Khan, R.
Farooq, M. A. Moawwez, and M. A. U.
R. Tariq, “A Quantitative Study of the
Impact of Organizational Culture,
Communication Management, and
Clarity in Project Scope on
Constructions’ Project Success with
Moderating Role of Project Manager’s
Competencies to Enhance Constructions
Management Practices,” Buildings, vol.
12, no. 11, p. 1856, Nov. 2022, doi:
10.3390/buildings12111856.
WSEAS TRANSACTIONS on SYSTEMS
DOI: 10.37394/23202.2024.23.44
Bryan Osvaldo, Ami Fitri Utami,
Mohammad Ichsan, Shaligram Pokharel
E-ISSN: 2224-2678
428
Volume 23, 2024
[46] S. S. Josyula, M. Suresh, and R. Raghu
Raman, “How to make intelligent
automation projects agile? Identification
of success factors and an assessment
approach,” International Journal of
Organizational Analysis, vol. 31, no. 5,
pp. 1461–1491, Oct. 2023, doi:
10.1108/IJOA-05-2021-2749.
[47] F. S. Altuwaijri and M. A. Ferrario,
“Factors affecting Agile adoption: An
industry research study of the mobile
app sector in Saudi Arabia,” Journal of
Systems and Software, vol. 190, p.
111347, Aug. 2022, doi:
10.1016/j.jss.2022.111347.
[48] K. Hayat, M. Hafeez, K. Bilal, and M.
S. Shabbir, “Interactive Effects of
Organizational Structure and Team
Work Quality on Project Success in
Project Based Non Profit
Organizations,” iRASD Journal of
Management, vol. 4, no. 1, pp. 84–103,
Mar. 2022, doi:
10.52131/jom.2022.0401.0064.
[49] M. K. M. Dahlan, N. Abdullah, and A.
I. H. Suhaimi, “The Propose
Organization Structure for Digital
Workplace,” Paper presented at the
2021 IEEE Symposium on Computers &
Informatics (ISCI) in Kuala Lumpur,
Malaysia, 2021,
https://doi.org/10.1109/ISCI51925.2021
.9633508.
[50] R. Adhikersa, W. Kumorotomo, A.
Djunaedi, and H. A. Hadna, “Impact of
Agile Organization and Leadership on
Employee Experience: Case Study
UPTD (Technical Implementing Service
Unit) Digital Service Center, Geospatial
Data and Information of West Java
Provincial Government (Jabar Digital
Service),” Populasi, vol. 30, no. 2, pp.
103–125, 2022,
https://doi.org/10.22146/jp.80191.
[51] J. A. Mena, “A Quantitative Approach
to the Organizational Design Problem,”
Florida International University, 2011.
doi: 10.25148/etd.FI11050610.
[52] S. D. Waskita and A. Hariadi, “Project
Performance Improvement Through
Project Maturity Level, Planning,
Implementation, Monitoring and
Controlling,” International Journal of
Innovative Science and Research
Technology , vol. 7, no. 7, 2022,
https://doi.org/10.5281/zenodo.7008302
[53] P. Philipp, F. Tobisch, and F. Matthes,
“Investigating the Adoption of Metrics
in Large-Scale Agile Software
Development,” Pacific Asia Conference
on Information Systems 2022 Taipei,
Jul. 2022, [Online].
https://aisel.aisnet.org/pacis2022/231/
(Accessed Date: November 26, 2024).
[54] F. Almeida and P. Carneiro, “Perceived
Importance of Metrics for Agile Scrum
Environments,” Information, vol. 14,
no. 6, p. 327, Jun. 2023, doi:
10.3390/info14060327.
[55] K. Wnuk and K. C. Maddila, “Agile and
lean metrics associated with
requirements engineering,” in
Proceedings of the 27th International
Workshop on Software Measurement
and 12th International Conference on
Software Process and Product
Measurement, New York, NY, USA:
ACM, Oct. 2017, pp. 33–40. doi:
10.1145/3143434.3143437.
[56] W. D. Piotrowicz, U. Ryciuk, and M.
Szymczak, “Lean and agile metrics.
Literature review and framework for
measuring leagile supply chain,”
International Journal of Productivity
and Performance Management, vol. 72,
no. 6, pp. 1560–1583, Jun. 2023, doi:
10.1108/IJPPM-10-2020-0560.
[57] S. Y. Ozornin, N. G. Terlyga, and D. B.
Shulgin, “Method for evaluating the
agile project management effectiveness
in information technology enterprises,”
2019, p. 020151. doi:
10.1063/1.5134302.
[58] A. A. M. Malik, F. M. Mustapha, M. N.
Sobri, A. F. N. Razak, M. N. M. Zaidi,
A. A. Shukri, and Z. L. A. M. Sham,
“Optimal Reliability and Validity of
Measurement Model in Confirmatory
Factor Analysis: Different Likert Point
Scale Experiment,” Journal of
Contemporary Issues and Thought, vol.
11, no. 1, pp. 105–112, 2021,
WSEAS TRANSACTIONS on SYSTEMS
DOI: 10.37394/23202.2024.23.44
Bryan Osvaldo, Ami Fitri Utami,
Mohammad Ichsan, Shaligram Pokharel
E-ISSN: 2224-2678
429
Volume 23, 2024
https://doi.org/10.37134/jcit.vol11.9.202
1.
[59] J. F. Hair and M. Sarstedt, “Factors versus
Composites: Guidelines for Choosing the Right
Structural Equation Modeling Method,”
Project Management Journal, vol. 50, no. 6,
pp. 619–624, Dec. 2019, doi:
10.1177/8756972819882132.
[60] J. F. Hair, J. J. Risher, M. Sarstedt, and C. M.
Ringle, “When to use and how to report the
results of PLS-SEM,” European Business
Review, vol. 31, no. 1, pp. 2–24, Jan. 2019, doi:
10.1108/EBR-11-2018-0203.
Contribution of Individual Authors to the
Creation of a Scientific Article (Ghostwriting
Policy)
The authors equally contributed in the present
research, at all stages from the formulation of the
problem to the final findings and solution.
Sources of Funding for Research Presented in a
Scientific Article or Scientific Article Itself
No funding was received for conducting this study.
Conflict of Interest
The authors have no conflict of interest to declare.
Creative Commons Attribution License 4.0
(Attribution 4.0 International, CC BY 4.0)
This article is published under the terms of the
Creative Commons Attribution License 4.0
https://creativecommons.org/licenses/by/4.0/deed.en_
US
WSEAS TRANSACTIONS on SYSTEMS
DOI: 10.37394/23202.2024.23.44
Bryan Osvaldo, Ami Fitri Utami,
Mohammad Ichsan, Shaligram Pokharel
E-ISSN: 2224-2678
430
Volume 23, 2024
APPENDIX
Table 1. Demographic samples (n = 156).
No
Demography
n
1
Gender
Female
102
Male
54
2
Educational Background
Bachelor's degree (S1)
94
Master's degree (S2)
12
Diploma
17
Others
33
3
APM Roles
Developer
37
Business Analyst
61
Product Manager
31
Solution Architect
23
Scrum Master
4
4
APM Experience
Below 1 year
42
1-5 years
98
6-10 years
12
More than 10 years
4
5
Industry Sector
Industrials
36
Financials
25
Communication Services
17
Consumer Discretionary
13
Energy
12
Information Technology
18
Others
35
WSEAS TRANSACTIONS on SYSTEMS
DOI: 10.37394/23202.2024.23.44
Bryan Osvaldo, Ami Fitri Utami,
Mohammad Ichsan, Shaligram Pokharel
E-ISSN: 2224-2678
431
Volume 23, 2024
Table 2. Measurement Model Analysis Result.
Construct
No. of
items
Cronbach's
Alpha
(0.6-0.9)
Composite
Reliability
(0.6-0.9)
AVE
(Average Variance
Extracted)
(>0.5)
Outer
Loadings
(>0.7)
Customer
Characteristics (CC)
7
0.890
0.897
0.601
0.709-0.838
Organizational
Influence (OI)
8
0.890
0.892
0.567
0.708-0.804
APM Effectiveness
(AF)
6
0.865
0.868
0.598
0.709-0.814
PM Methodology
(PM)
6
0.856
0.893
0.581
0.725-0.810
Table 3. Correlation Matrix (HTMT Ratio).
APM
Effectiveness
Customer
Characteristics
Organizational
Influence
PM
Methodology
(PM)
x
(CC)
APM Effectiveness
Customer
Characteristics
0.613
Organizational
Influence
0.885
0.695
PM Methodology
0.754
0.76
0.688
(PM) x (CC)
0.322
0.197
0.293
0.423
Table 4. Coefficient of determination and predictive relevance.
Construct
R-square
Q-square
APM Effectiveness
0.663
0.381
Organizational Influence
0.408
0.373
Table 5. Effect Size (F2).
Relationship of Construct to APM
Effectiveness
APM Effectiveness
Customer Characteristics
0.002
Organizational Influence
0.600
PM Methodology
0.119
PM Methodology x Customer
Characteristics
0.001
WSEAS TRANSACTIONS on SYSTEMS
DOI: 10.37394/23202.2024.23.44
Bryan Osvaldo, Ami Fitri Utami,
Mohammad Ichsan, Shaligram Pokharel
E-ISSN: 2224-2678
432
Volume 23, 2024
Table 6. PLS Predict
Items
RMSE PLS
RMSE LM
Comparison
AF3
0.747
0.711
Bigger
AF4
0.813
0.832
Smaller
AF5
0.723
0.738
Smaller
AF6
0.742
0.755
Smaller
AF7
0.745
0.748
Smaller
AF8
0.808
0.83
Smaller
OI10
0.781
0.805
Smaller
OI11
0.808
0.825
Smaller
OI3
0.815
0.818
Smaller
OI4
0.789
0.813
Smaller
OI5
0.808
0.854
Smaller
OI7
0.81
0.804
Bigger
OI8
0.855
0.901
Smaller
OI9
0.74
0.781
Smaller
Table 7. Variance Inflation Factor (VIF) results from the inner model
Constructs
VIF
Customer Characteristics -> APM Effectiveness
2.168
Customer Characteristics -> Organizational Influence
1.000
Organizational Influence -> APM Effectiveness
1.892
PM Methdology -> APM Effectiveness
2.215
PM Methodology x Customer Characteristics -> APM
Effectiveness
1.209
Table 8. Hypotheses Test Result
Hypotheses
Structural Paths
Standardized
Coefficient
T-
value
P-value
Hypotheses
Result
H1
Customer Characteristics ->
APM effectiveness
-0.036
0.488
0.626
Not
supported
H2
Customer Characteristics ->
Organizational Influence
0.639
13.056
0.001
Supported
H3
Organizational Influence ->
APM effectiveness
0.618
4.786
0.001
Supported
H4
PM Methodology x Customer
Characteristics -> APM
effectiveness
-0.017
0.411
0.681
Not
supported
WSEAS TRANSACTIONS on SYSTEMS
DOI: 10.37394/23202.2024.23.44
Bryan Osvaldo, Ami Fitri Utami,
Mohammad Ichsan, Shaligram Pokharel
E-ISSN: 2224-2678
433
Volume 23, 2024
Fig. 2: Structural Model Analysis containing 5000 subsamples
WSEAS TRANSACTIONS on SYSTEMS
DOI: 10.37394/23202.2024.23.44
Bryan Osvaldo, Ami Fitri Utami,
Mohammad Ichsan, Shaligram Pokharel
E-ISSN: 2224-2678
434
Volume 23, 2024