2020
A Data Analysis Platform to Evaluate Performance During Software Development Process
Silvana de Gyves Avila, Patricia Ortegon Cano, Ayrton Mondragon Mejia, Ismael Solis Moreno, Arianne Navarro Lepe, Gloria Eva Zagal Dominguez
RISTI: Revista Iberica de Sistemas e Tecnologias de Informacao pp. 36, 50-64, 2020
Abstract software, multimedia, library science, computer science
Performance is one of the most important parameters to consider during the software development process. It is used as competitive advantage among similar solutions. Assuring expected performance levels is not trivial since it requires to run exhaustive tests and the analysis of Big Data. Normally, companies rely on commercial solutions to produce performance analytics. However, these require a significant effort to be adapted to particular performance use cases. In this paper, we describe DDP, a platform to analyze performance testing data. DDP uses Big Data technology to collect, store, process and analyze performance results in an integrated way. We demonstrated the successful use of DDP evaluating the performance of Spectrum Scale, a software defined storage solution. We illustrate the use of DDP analyzing data from performance regression tests to verify and validate the quality of new versions build during the development process.
doi
software, multimedia, library science, computer science
2019
A Data Driven Platform for Improving Performance Assessment of Software Defined Storage Solutions
Silvana De Gyves Avila, Patricia Ortegon Cano, Ayrton Mondragon Mejia, Ismael Solis Moreno, Arianne Navarro Lepe
International Conference on Software Process Improvement, 266-275, 2019
Abstract use case, software defined storage, software quality, software engineering, software development process, software, exploit, computer science, big data, analytics
Performance is one of the most important dimensions to consider for software quality. It is normally used as the principal competitive advantage offered between similar solutions. Ensuring certain performance levels is not trivial for software companies, since it requires exhaustive test runs and massive data analysis. Normally, companies rely on commercial analytics packages to produce performance insights. However, these are for general purposes and require significant effort to be adapted to particular data requirements use cases. In this paper we introduce DDP, a highly scalable data driven performance platform to analyze and exploit performance data for software defined storage solutions. DDP employs big data and analytics technology to collect, store and process performance data in an efficient and integrated way. We have demonstrated the successful application of DDP for Spectrum Scale, a software defined storage solution, where we have been able to implement performance regression data analysis to validate the performance consistency of new produced builds.
doi
use case, software defined storage, software quality, software engineering, software development process, software, exploit, computer science, big data, analytics
2018
Color uniformity descriptor: An efficient contextual color representation for image indexing and retrieval
Carolina Reta, Jose A. Cantoral-Ceballos, Ismael Solis-Moreno, Jesus A. Gonzalez, Rogelio Alvarez-Vargas, Nery Delgadillo-Checa
Journal of Visual Communication and Image Representation54, 39-50, 2018
Abstract throughput, search engine indexing, robustness, pixel, pattern recognition, mathematics, histogram, computer vision, complex data type, color representation, color image, artificial intelligence
Abstract 1 2 Color is a rich source of visual information for the effective characterization of image content. The recognition of texture or shape elements in images is strongly associated with the analysis of the image color layout. This paper presents a contextual color descriptor designed especially to be applied to CBIR tasks in heterogeneous image databases. The proposed color uniformity descriptor (CUD) clusters perceptually similar image color regions according to the uniformity analysis of their neighbor pixels. CUD produces vast color image details with a thin histogram, whilst preserving the balance between uniqueness and robustness. CUD is computationally efficient and can achieve high precision and throughput rates when used in CBIR. Experimental results show that CUD performs comparably against local features and multiple features state-of-the-art approaches that require more complex data manipulation. Results demonstrate that CUD provides strong image discrimination even in the presence of significant content variation.
doi
throughput, search engine indexing, robustness, pixel, pattern recognition, mathematics, histogram, computer vision, complex data type, color representation, color image, artificial intelligence
Improving content-based image retrieval for heterogeneous datasets using histogram-based descriptors
Carolina Reta, Ismael Solis-Moreno, Jose A. Cantoral-Ceballos, Rogelio Alvarez-Vargas, Paul Townend
Multimedia Tools and Applications 77(7), 8163-8193, 2018
Abstract visual word, pattern recognition, local binary patterns, image texture, image retrieval, contextual image classification, content based image retrieval, computer vision, computer science, cluster analysis, automatic image annotation, artificial intelligen
Image content analysis plays a key role in areas such as image classification, clustering, indexing, retrieving, and object and scene recognition. However, although several image content descriptors have been proposed in the literature, their low performance score or high computational cost makes them unsuitable for content-based image retrieval on large datasets. This paper presents an efficient content-based image retrieval approach that uses histogram-based descriptors to represent color, edge, and texture features, and a k-nearest neighbor classifier to retrieve the best matches for query images. The compactness and speed of the proposed descriptors allow their application in heterogeneous photographic collections whilst showing strong image discrimination in the presence of significant content variation. Experimentation was conducted on four different image collections using four distance metrics. The results show that the proposed approach consistently achieves noteworthy mean average precision, recall, and precision measures. It outperforms state-of-the-art approaches based on the MPEG 7 descriptors (SCD, CLD, and EHD), whilst producing comparable results to those achieved by novel SIFT-based and SURF-based approaches that require more complex data manipulation.
doi
visual word, pattern recognition, local binary patterns, image texture, image retrieval, contextual image classification, content based image retrieval, computer vision, computer science, cluster analysis, automatic image annotation, artificial intelligen
2016
Tolerating Transient Late-Timing Faults in Cloud-Based Real-Time Stream Processing
Peter Garraghan, Stuart Perks, Xue Ouyang, David McKee, Ismael Solis Moreno
2016 IEEE 19th International Symposium on Real-Time Distributed Computing (ISORC), pp. 108-115
Abstract stream processing, software fault tolerance, real time computing, quality of service, fault tolerance, distributed computing, data prediction, crash, computer science, cloud computing, cpu time
Real-time stream processing is a frequently deployed application within Cloud datacenters that is required to provision high levels of performance and reliability. Numerous fault-tolerant approaches have been proposed to effectively achieve this objective in the presence of crash failures. However, such systems struggle with transient late-timing faults -- a fault classification challenging to effectively tolerate -- that manifests increasingly within large-scale distributed systems. Such faults represent a significant threat towards minimizing soft real-time execution of streaming applications in the presence of failures. This work proposes a fault-tolerant approach for QoS-aware data prediction to tolerate transient late-timing faults. The approach is capable of determining the most effective data prediction algorithm for imposed QoS constraints on a failed stream processor at run-time. We integrated our approach into Apache Storm with experiment results showing its ability to minimize stream processor end-to-end execution time by 61% compared to other fault-tolerant approaches. The approach incurs 12% additional CPU utilization while reducing network usage by 44%.
doi
stream processing, software fault tolerance, real time computing, quality of service, fault tolerance, distributed computing, data prediction, crash, computer science, cloud computing, cpu time
2015
An Approach to Improve Mouth-State Detection to Support the ICAO Biometric Standard for Face Image Validation
Salvador Coronel Castellanos, Ismael Solis Moreno, Jose A. Cantoral Ceballos, Rogelio Alvarez Vargas, Pedro L. Martinez Quintal
2015 International Conference on Mechatronics, Electronics and Automotive Engineering (ICMEAE), pp. 35-40
Abstract three dimensional face recognition, pattern recognition, object class detection, image segmentation, image processing, gesture, facial recognition system, face detection, computer vision, computer science, biometrics, artificial intelligence
Face image analysis continues as an ongoing challenge in biometrics and image processing due to the state variations of facial elements. In this context, the mouth-state plays a fundamental role because its impact on the perception of facial gestures. Current work on mouth-state detection is mainly focused on the creation of classifiers derived from large training datasets. This technique requires extensive training sessions and its results entirely rely on the quality of datasets and learning methods. This paper describes an original approach for detecting mouth-state to support the ICAO standard for face image validation. The proposed approach reduces the error margins by considering face proportions for image segmentation and estimates the magnitude of mouth aperture by conducting an analysis of skin color. Experimentation demonstrates improvements by 21% on the correct detection of mouth-state by slightly affecting the processing time in comparison to the classifiers approach.
doi
three dimensional face recognition, pattern recognition, object class detection, image segmentation, image processing, gesture, facial recognition system, face detection, computer vision, computer science, biometrics, artificial intelligence
2014
Characterizing and exploiting heterogeneity for enhancing energy-efficiency of cloud datacenters
Ismael Solis Moreno
2014
Abstract workload, virtualization, server, real time computing, quality of service, exploit, engineering, energy consumption, efficient energy use, distributed computing, cloud computing
Cloud Computing environments are composed of large and powerconsuming datacenters designed to support the elasticity required by their customers. The adoption of Cloud Computing is rapidly growing since it promises cost reductions for customers in comparison with permanentinvestments in traditional datacenters. However, for Cloud providers, energy consumption represents a serious problem since they have to deal with the increasing demand and diverse Quality of Service requirements.Contemporary energy-efficient Cloud approaches exploit the advantages of virtualization to maximize the use of physical resources and minimize the number of active servers.A major problem not considered by current Cloud resource management schemes is that of the inherent heterogeneity of customer, workload and server types in multi-tenant environments. This is an issue when improving energy-efficiency, as co-location of specific workload types may result in strong contention for the physical resources. This then affects the resource consumption patterns and therefore the energy-efficiency of virtualized servers. In addition, because of the on-demand self-service characteristic of the Cloud model, different types of customers tend to highly overestimate the amount of required resources. This creates a non-negligible amount ofunderutilized servers that affects the energy-efficiency of the datacenter.This thesis analyzes a production Cloud environment to determine the characteristics of the heterogeneous customer, workload and server types, and proposes a novel way to exploit such heterogeneity in order to improveenergy-efficiency through two mechanisms. The first improves energyefficiency by co-locating diverse workload types according to the minimum level of produced interference in a heterogeneous pool of servers. Thesecond mitigates the waste generated by customer overestimation by dynamically overallocating resources based on heterogeneous customer profiles and the levels of produced interference. The evaluation of the proposed mechanisms demonstrates that considering the heterogeneity of elements in a Cloud environment supports the effective improvement of the datacenter energy-efficiency and the performance of individual workloads.
workload, virtualization, server, real time computing, quality of service, exploit, engineering, energy consumption, efficient energy use, distributed computing, cloud computing
An Analysis of Failure-Related Energy Waste in a Large-Scale Cloud Environment
Peter Garraghan, Ismael Solis Moreno, Paul Townend, Jie Xu
IEEE Transactions on Emerging Topics in Computing 2(2), 166-180, 2014
Abstract server, real time computing, provisioning, operational costs, energy consumption, efficient energy use, dependability, computer science, cloud systems, cloud computing
Cloud computing providers are under great pressure to reduce operational costs through improved energy utilization while provisioning dependable service to customers; it is therefore extremely important to understand and quantify the explicit impact of failures within a system in terms of energy costs. This paper presents the first comprehensive analysis of the impact of failures on energy consumption in a real-world large-scale cloud system (comprising over 12 500 servers), including the study of failure and energy trends of the spatial and temporal environmental characteristics. Our results show that 88% of task failure events occur in lower priority tasks producing 13% of total energy waste, and 1% of failure events occur in higher priority tasks due to server failures producing 8% of total energy waste. These results highlight an unintuitive but significant impact on energy consumption due to failures, providing a strong foundation for research into dependable energy-aware cloud computing.
doi
server, real time computing, provisioning, operational costs, energy consumption, efficient energy use, dependability, computer science, cloud systems, cloud computing
Analysis, Modeling and Simulation of Workload Patterns in a Large-Scale Utility Cloud
Ismael Solis Moreno, Peter Garraghan, Paul Townend, Jie Xu
ieee international conference on cloud computing technology and science, pp. 208-221, 2014
Abstract workload, statistical hypothesis testing, resource management, real time computing, quality of service, distributed computing, computer science, cloudsim, cloud testing, cloud computing, behavioral pattern
Understanding the characteristics and patterns of workloads within a Cloud computing environment is critical in order to improve resource management and operational conditions while Quality of Service (QoS) guarantees are maintained. Simulation models based on realistic parameters are also urgently needed for investigating the impact of these workload characteristics on new system designs and operation policies. Unfortunately there is a lack of analyses to support the development of workload models that capture the inherent diversity of users and tasks, largely due to the limited availability of Cloud tracelogs as well as the complexity in analyzing such systems. In this paper we present a comprehensive analysis of the workload characteristics derived from a production Cloud data center that features over 900 users submitting approximately 25 million tasks over a time period of a month. Our analysis focuses on exposing and quantifying the diversity of behavioral patterns for users and tasks, as well as identifying model parameters and their values for the simulation of the workload created by such components. Our derived model is implemented by extending the capabilities of the CloudSim framework and is further validated through empirical comparison and statistical hypothesis tests. We illustrate several examples of this works practical applicability in the domain of resource management and energy-efficiency.
doi
workload, statistical hypothesis testing, resource management, real time computing, quality of service, distributed computing, computer science, cloudsim, cloud testing, cloud computing, behavioral pattern
2013
An Analysis of Performance Interference Effects on Energy-Efficiency of Virtualized Cloud Environments
Renyu Yang, Ismael Solis Moreno, Jie Xu, Tianyu Wo
2013 IEEE 5th International Conference on Cloud Computing Technology and Science, pp. 112-119
Abstract workload, server, resource management, resource allocation, real time computing, quality of service, interference, distributed computing, data center, computer science, cloud computing
Co-allocated workloads in a virtualized computing environment often have to compete for resources, thereby suffering from performance interference. While this phenomenon has a direct impact on the Quality of Service provided to customers, it also changes the patterns of resource utilization and reduces the amount of work per Watt consumed. Unfortunately, there has been only limited research into how performance interference affects energy-efficiency of servers in such environments. In reality, there is a highly dynamic and complicated correlation among resource utilization, performance interference and energy-efficiency. This paper presents a comprehensive analysis that quantifies the negative impact of performance interference on the energy-efficiency of virtualized servers. Our analysis methodology takes into account the heterogeneous workload characteristics identified from a real Cloud environment. In particular, we investigate the impact due to different workload type combinations and develop a method for approximating the levels of performance interference and energy-efficiency degradation. The proposed method is based on profiles of pair combinations of existing workload types and the patterns derived from the analysis. Our experimental results reveal a non-linear relationship between the increase in interference and the reduction in energy-efficiency as well as an average precision within +/-5% of error margin for the estimation of both parameters. These findings provide vital information for research into dynamic trade-offs between resource utilization, performance, and energy-efficiency of a data center.
doi
workload, server, resource management, resource allocation, real time computing, quality of service, interference, distributed computing, data center, computer science, cloud computing
Improved energy-efficiency in cloud datacenters with interference-aware virtual machine placement
Ismael Solis Moreno, Renyu Yang, Jie Xu, Tianyu Wo
2013 IEEE Eleventh International Symposium on Autonomous Decentralized Systems (ISADS), pp. 1-8
Abstract workload, virtualization, virtual machine, resource efficiency, real time computing, quality of service, interference, efficient energy use, distributed computing, computer science, cloud computing
Virtualization is one of the main technologies used for improving resource efficiency in datacenters; it allows the deployment of co-existing computing environments over the same hardware infrastructure. However, the co-existing of environments along with management inefficiencies often creates scenarios of high-competition for resources between running workloads, leading to performance degradation. This phenomenon is known as Performance Interference, and introduces a non-negligible overhead that affects both a datacenters Quality of Service and its energy-efficiency. This paper introduces a novel approach to workload allocation that improves energy-efficiency in Cloud datacenters by taking into account their workload heterogeneity. We analyze the impact of performance interference on energy-efficiency using workload characteristics identified from a real Cloud environment, and develop a model that implements various decision-making techniques intelligently to select the best workload host according to its internal interference level. Our experimental results show reductions in interference by 27.5% and increased energy-efficiency up to 15% in contrast to current mechanisms for workload allocation.
doi
workload, virtualization, virtual machine, resource efficiency, real time computing, quality of service, interference, efficient energy use, distributed computing, computer science, cloud computing
An Approach for Characterizing Workloads in Google Cloud to Derive Realistic Resource Utilization Models
I. S. Moreno, P. Garraghan, P. Townend, Jie Xu
2013 IEEE Seventh International Symposium on Service-Oriented System Engineering, pp. 49-60
Abstract workload, server, real time computing, margin of error, distributed computing, computer science, cloud testing, cloud data center, cloud data, cloud computing, behavioral pattern
Analyzing behavioral patterns of workloads is critical to understanding Cloud computing environments. However, until now only a limited number of real-world Cloud data center trace logs have been available for analysis. This has led to a lack of methodologies to capture the diversity of patterns that exist in such datasets. This paper presents the first large-scale analysis of real-world Cloud data, using a recently released dataset that features traces from over 12,000 servers over the period of a month. Based on this analysis, we develop a novel approach for characterizing workloads that for the first time considers Cloud workload in the context of both user and task in order to derive a model to capture resource estimation and utilization patterns. The derived model assists in understanding the relationship between users and tasks within workload, and enables further work such as resource optimization, energy-efficiency improvements, and failure correlation. Additionally, it provides a mechanism to create patterns that randomly fluctuate based on realistic parameters. This is critical to emulating dynamic environments instead of statically replaying records in the trace log. Our approach is evaluated by contrasting the logged data against simulation experiments, and our results show that the derived model parameters correctly describe the operational environment within a 5% of error margin, confirming the great variability of patterns that exist in Cloud computing.
doi
workload, server, real time computing, margin of error, distributed computing, computer science, cloud testing, cloud data center, cloud data, cloud computing, behavioral pattern
2012
Neural Network-Based Overallocation for Improved Energy-Efficiency in Real-Time Cloud Environments
Ismael Solis Moreno, Jie Xu
2012 IEEE 15th International Symposium on Object/Component/Service-Oriented Real-Time Distributed Computing, pp. 119-126
Abstract virtual machine, server, resource management, resource allocation, real time computing, provisioning, green computing, distributed computing, data center, computer science, cloud computing
This paper introduces a dynamic resource provisioning mechanism for over allocating the capacity of Cloud data centers based on customer resource utilization patterns. The proposed mechanism reduces the impact on Real-Time constraints while improvements on the overall energy-efficiency are sought. The main idea is to exploit the resource utilization patterns of each customer for smartly under allocating resources to the requested Virtual Machines. This reduces the waste produced by frequent overestimations and increases the data center availability. Consequently, it creates the opportunity to host additional Virtual Machines in the same computing infrastructure improving its energy-efficiency. In order to mitigate the negative effect on deadlines, the proposed over allocation service implements a multiplayer Neural Network to anticipate the resource usage patterns based on historical data. Additionally, a compensation mechanism for adjusting the resource allocation in cases of unexpected higher demand is also described. The experiments contrast the proposed approach against traditional "Dynamic Resource Resizing" energy-aware mechanisms and also to our previous work that implements Low-Pass-Filter as predictor. Results demonstrate meaningful improvements in energy-efficiency while time constraints are slightly affected.
doi
virtual machine, server, resource management, resource allocation, real time computing, provisioning, green computing, distributed computing, data center, computer science, cloud computing
2011
Customer-aware resource overallocation to improve energy efficiency in realtime Cloud Computing data centers
Ismael Solis Moreno, Jie Xu
2011 IEEE International Conference on Service-Oriented Computing and Applications (SOCA), pp. 1-8
Abstract scalability, resource allocation, real time computing, quality of service, provisioning, green computing, energy conservation, efficient energy use, distributed computing, computer science, cloud computing
Energy efficiency is becoming a very important concern for Cloud Computing environments. These are normally composed of large and power consuming data centers to provide the required elasticity and scalability to their customers. In this context, many efforts have been developed to balance the loads at host level. However, determining how to maximize the resources utilization at Virtual Machine (VM) level still remains as a big challenge. This is mainly driven by very dynamic workload behaviors and a wide variety of customers resource utilization patterns. This paper introduces a dynamic resource provisioning mechanism to overallocate the capacity of real-time Cloud data centers based on customer utilization patterns. Furthermore, its impact on the trade-off between energy efficiency and SLA fulfillment is analyzed. The main idea is to exploit the resource utilization patterns of each customer to decrease the waste produced by resource request overestimations. This creates the opportunity to allocate additional VMs in the same host incrementing its energy efficiency. Nevertheless, this also increases the risk of QoS affectations. The proposed model considers SLA deadlines, predictions based on historical data, and dynamic occupation to determine the amount of resources to overallocate for each host. In addition, a compensation mechanism to adjust resource allocation in cases of underestimation is also described. In order to evaluate the model, simulation experimentation was conducted. Results demonstrate meaningful improvements in energy-efficiency while SLA-deadlines are slightly impacted. However, they also point the importance of strongest compensation policies to reduce availability violations especially during peak utilization periods.
doi
scalability, resource allocation, real time computing, quality of service, provisioning, green computing, energy conservation, efficient energy use, distributed computing, computer science, cloud computing
Energy-Efficiency in Cloud Computing Environments: Towards Energy Savings without Performance Degradation
Jie Xu, Ismael Solis Moreno
International Journal of Cloud Applications and Computing archive 1(1), 17-33, 2011
Abstract utility computing, risk analysis, real time computing, green computing, energy consumption, efficient energy use, economic problem, computer science, cloud computing, central management, boosting
Due to all the pollutants generated during its production and the steady increases in its rates, energy consumption is causing serious environmental and economic problems. In this context, the growing use and adoption of ICTs is being highlighted not only as one as the principal problem sources but also as one of the principal areas that could help in the problems reduction. Cloud computing is an emerging model for distributed utility computing and is being considered as an attractive opportunity for saving energy through central management of computational resources. To be successful, the design of energy-efficient mechanisms must start playing a mayor role. This paper argues the importance of energy-efficient mechanisms within cloud data centers and remarks on the significance of the "energy-performance" relationship in boosting the adoption of these mechanisms in real scenarios. It provides an analysis of the current approaches and the outline of key opportunities that need to be addressed to improve the "energy-performance" relationship in this promising model.
doi
utility computing, risk analysis, real time computing, green computing, energy consumption, efficient energy use, economic problem, computer science, cloud computing, central management, boosting
2006
Using case-based reasoning for improving precision and recall in web services selection
Olivia G. Fragoso Diaz, Rene Santaolaya Salgado, Ismael Solis Moreno
International Journal of Web and Grid Services 2(3), 306-330, 2006
Abstract world wide web, web standards, web service, web modeling, web development, ws policy, ws i basic profile, semantic web stack, data mining, data web, computer science
Web services are currently one of the main technologies employed to create a systematic and extensible framework for application development. However, due to the large number of web services that exist nowadays, locating one or several web services to fulfil the functional requirements of a user or an organisation, is a complex and time consuming activity for application developers, also reducing their productivity. One possible solution for this problem is the implementation of a semantic component, structured as a library and populated with cases which represent web services in such a way that it may extend the functionality of the existing web services directories. This paper describes a model for searching and selecting web services supported by Case-Based Reasoning (CBR). Results showed that the extension to Universal Description, Discovery and Integration (UDDI) yielded 100% recall while precision improved significantly, depending some times on the way the user queried the system.
doi
world wide web, web standards, web service, web modeling, web development, ws policy, ws i basic profile, semantic web stack, data mining, data web, computer science
Searching and selecting web services using case based reasoning
Olivia Graciela Fragoso Diaz, Rene Santaolaya Salgado, Ismael Solis Moreno, Guillermo Rodriguez Ortiz
ICCSA06 Proceedings of the 2006 international conference on Computational Science and Its Applications - Volume Part IV, pp. 50-57
Abstract world wide web, web service, web page, web modeling, ws policy, ws i basic profile, services computing, semantic web stack, data web, computer science
Web services are currently one of the main technologies employed to create a systematic and extensible framework for application development. This is done by means of allowing the interaction among the applications of an organization. However, due to the large number of web services that may exist nowadays, locating one or several web services to fulfill the functional requirements of a user, an organization or a business entity, is a complex and time consuming activity for application developers. It also reduces their productivity. One possible solution for this problem is the implementation of a semantic component, structured as a library and populated with cases represented by web services in such a way that it may extend the functionality of the existing web services directories. The semantic component must provide a mechanism for classifying and selecting web services based on their functionality and supporting the search of WSDL description files of selected web services in a non sequential order within the directories. This paper describes a model for searching and selecting web services in UDDI directories supported by case based reasoning. Advantages and limitations of the model are also described.
doi
world wide web, web service, web page, web modeling, ws policy, ws i basic profile, services computing, semantic web stack, data web, computer science
