FE - DI | Dissertações de Mestrado e Teses de Doutoramento
Permanent URI for this collection
Browse
Recent Submissions
- How Can Deep Learning Aid Human Behavior Analysis?Publication . Roxo, Tiago Filipe Dias dos Santos ; Proença, Hugo Pedro Martins Carriço; Inácio, Pedro Ricardo MoraisWith the increase of available surveillance data and robustness of state-of-the-art deep learning models, various recent research topics focus on human biometric assessment, tracking and person re-identification. However, one other area of work not extensively explored that can combine surveillance and visual-based models is assessing human behavior. The lack of work in this topic is not surprising given the inherent difficulties on categorizing human behavior in such conditions, in particular without subject cooperation. Based on the psychology literature, human behavior analysis typically requires controlled experimental environments, with subject cooperation and assessing features via grid-based survey. As such, it is not clear on how deep learning models can aid psychology experts in human behavior analysis, which is where this thesis intents to contribute to the body of knowledge. We extensively review psychology literature to define a set of features that have been proven as influential towards human behavior and that can be assessed via camera in surveillance-like conditions. This way, we define human behavior via subject profiling using seven behavioral features: interaction, relative position, clothing, soft biometrics, subject proximity, pose, and use of handheld devices. Note that this analysis does not categorize human behavior into specific states (e.g. aggressive, depressive) but rather creates a set of features that can be used to profile subjects, usable to aid/complement behavioral experts and to compare behavioral traits between subjects in a scene. Furthermore, to motivate the development of works in these areas, we review state-of-the-art approaches and datasets to highlight the limitation of certain areas and discuss the topics worth exploring for future works. After defining a set of behavioral features, we start by exploring the limitation of current biometric models in surveillance conditions, in particular the resilience of gender inference approaches. We demonstrate that these models underperform in surveillance-like data, using PAR datasets, highlighting the limitations of training in cooperative settings to perform in wilder conditions. Supported by the findings of our initial experiments, complementing face and body information arouse as a viable strategy to increase model robustness in these conditions, which lead us to design and propose a new model for wild gender inference based on this premise. This way, we extend the knowledge of an extensive discussed literature topic (gender classification) by exploring its application in settings where current models do not typically perform (surveillance). We also explore the topic of human interaction, namely Active Speaker Detection, in particular towards more uncooperative scenarios such as surveillance conditions. Contrary to the gender/biometrics topic, this is a lesser explored area where works are mainly based on assessing active speakers via face and audio information in cooperative conditions and with good audio and image quality (movie settings). As such, to clearly demonstrate the limitations of state-of-the-art ASD models we start by creating a wilder ASD dataset (WASD), composed of different categories with increasing challenges towards ASD, namely with audio and image quality degradation, and containing uncooperative subjects. This dataset highlighted the limitations of current models to deal with unconstrained scenarios (e.g. surveillance conditions), while also displaying the importance of body information in conditions where audio quality is subpar and face access is not guaranteed. Following this premise, we design the first model that complements audio, face, and body information to achieve state-of-the-art performance in challenging conditions, in particular surveillance settings. Furthermore, this model also proposed a novel way to combine data via SE blocks, which allowed to provide reasoning behind model’s decision by visual interpretability. The use of SE blocks was also extended to other models and ASD-related areas to highlight the viability of this approach for model-agnostic interpretability. Although this initial model was superior to the state-of-the-art in challenging data, its performance in cooperative settings was not as robust. As such, we develop a new model that simultaneously combines face and body information in visual data extraction which, in conjunction with pretraining in challenging data, leads to state-of-the-art performance in both cooperation and challenging conditions (such as surveillance settings). These works pave a new way to assess human interaction in more challenging data and with model interpretability, serving as baselines for future works.
- Improving the Robustness of Demonstration LearningPublication . Correia, André Rosa de Sousa Porfírio; Alexandre, Luís Filipe Barbosa de AlmeidaWith the fast improvement of machine learning, Reinforcement Learning (RL) has been used to automate human tasks in different areas. However, training such agents is difficult and restricted to expert users. Moreover, it is mostly limited to simulation environments due to the high cost and safety concerns of interactions in the real world. Demonstration Learning is a paradigm in which an agent learns to perform a task by imitating the behavior of an expert shown in demonstrations. It is a relatively recent area in machine learning, but it is gaining significant traction due to having tremendous potential for learning complex behaviors from demonstrations. Learning from demonstration accelerates the learning process by improving sample efficiency, while also reducing the effort of the programmer. Due to learning without interacting with the environment, demonstration learning can allow the automation of a wide range of real world applications such as robotics and healthcare. Demonstration learning methods still struggle with a plethora of problems. The estimated policy is reliant on the coverage of the data set which can be difficult to collect. Direct imitation through behavior cloning learns the distribution of the data set. However, this is often not enough and the methods may struggle to generalize to unseen scenarios. If the agent visits out-of-distribution cases, not only will it not know what to do, but the consequences in the real world can be catastrophic. Because of this, offline RL methods try to specifically reduce the distributional shift. In this thesis, we focused on proposing novel methods to tackle some of the open problems in demonstration learning. We start by introducing the fundamental concepts, methodologies, and algorithms that underpin the proposed methods in this thesis. Then, we provide a comprehensive study of the state-of-the-art of Demonstration Learning methods. This study allowed us to understand existing methods and expose the open problems which motivate this thesis. We then developed five methods that push improve upon the state-of-the-art and solve different problems. The first method proposes to tackle the context problem, where policies are restricted to the context in which they were trained. We propose a method to learn context-invariant image representations with contrastive learning, by making use of a multi-view demonstration data set. We show that these representations can be used in lieu of the original images to learn a policy with standard reinforcement learning algorithms. This work also contributed with benchmark environment and a demonstration data set. Next, we tackled the potential of combining reinforcement learning with demonstration learning to cover the weaknesses of both paradigms. Specifically, we developed a method to improve the safety of reinforcement learning agents during their learning process. The proposed method makes use of a demonstration data set with safe and unsafe trajectories. Before each interaction, the method evaluates the trajectory and stops it if deems it unsafe. The method was used to augment state-of-theart reinforcement learning methods, and it reduced the crash rate significantly which also resulted in a slight increase in performance. In the following work, we acknowledged the significant strides made in sequence modelling and their impact in a plethora of machine learning problems. We noticed that these methods had recently been applied to demonstration learning. However, the state-of-the-art method was reliant on task knowledge and user interaction to perform. We proposed a hierarchical method which identifies important states in each demonstration, and uses them to guide the sequence model. The result is a method that is task and user independent but also achieves better performance than the previous state-of-the-art. Next, we made use of the novel Mamba architecture to improve upon the previous sequence modelling method. By replacing the Transformer architecture with the Mamba, we proposed two methods that reduce the complexity, and inference time while also improving the performance. Finally, we apply demonstration learning to under-explored applications. Specifically, we apply demonstration learning to teach an agent to dance to music. We describe the insight of modelling the task of learning to dance as a translation task, where the agent learns to translate from the language of music to the language of dance. We used the previous experience resulted from the two sequence modelling methods to propose two variants: using the Transformer or the Mamba architectures. The method modifies the standard sequence modelling architecture to process sequences of audio features and translate them to dance poses. Results show that the method can translate diverse and unseen music to high-quality dance motions coherent within the genre. Results obtained by the proposed methods advance the state-of-the-art in Demonstration Learning and provide solutions to open problems in the field. All the proposed methods were evaluated against state-of-the-art baselines and evaluated on several tasks and diverse data sets, improving the performance and tackling their respective problems.
- Detection of Stealthy Distributed Denial of Service Attacks Using Artificial Intelligence MethodsPublication . Rios, Vinícius de Miranda; Freire, Mário Marques; Magoni, DamienDistributed Denial of Service (DDoS) attacks have been used to disrupt various online activities. The significant traffic volume of these distributed attacks has enabled the identification of signatures and behavior profiles that fostered the development of detection mechanisms for mitigating these attacks. However, as new attack types emerge, such as low-rate Denial of Service (DoS) attacks, new detection mechanisms need to be developed to combat these evolving threats effectively. Many detection mechanisms rely primarily on statistical analysis to identify low-rate DoS attacks in data traffic. However, these methods often exhibit a high rate of false negatives and are only applicable to small-scale data. Artificial intelligence techniques have been widely employed in various fields, including social network analysis and disease monitoring, and have gradually gained prominence in the field of cybersecurity in recent years. This thesis focuses on studying and developing detection mechanisms that exhibit effective performance against two specific types of low-rate DoS attacks: the Reduction of Quality (RoQ) attack and the Slowloris attack. For the RoQ attack, we examine the traffic transmission format to create a similar one, as there is no existing software capable of generating this type of attack traffic on the internet. For the Slowloris attack, we utilized free and open-source software specifically developed for this purpose. Subsequently, we analyze the traffic from both attacks and extract features that can be used by detection mechanisms. In this thesis, two approaches have been developed for classifying and detecting RoQ and Slowloris attacks: one approach is based on the separate use of a set of traditional Machine Learning (ML) algorithms and the second approach is based on fuzzy logic plus one traditional ML algorithm (that previously led to good classification results) and Euclidean distance. For the RoQ attack detection, the first approach uses eleven separate machine learning algorithms, namely K-Nearest Neighbors (K-NN), Multilayer Perceptron Neural Network (MLP), Support Vector Machine (SVM), Multinomial Naive Bayes (MNB), Gaussian Naive Bayes (GNB), Decision Tree (DT), Random Forest (RF), Gradient Boosting (XGB), Logistic Regression (LR), AdaBoost, and Light Gradient Boosting Machine (LGBM), while the second approach consists in our proposed method which combines fuzzy logic, the MLP algorithm, and the Euclidean distance method. For the Slowloris attack detection, the first approach utilizes nine machine learning algorithms, namely KNN, GNB, MLP, SVM, DT, MNB, RF, XGB, and LGBM, while the second approach consists in our proposed method which combines fuzzy logic, the RF algorithm, and the Euclidean distance method. Both approaches utilize previously selected features to classify the data traffic as either attack traffic or legitimate traffic. The obtained results show that some ML algorithms (namely MLP and RF) as well as our approach based on fuzzy logic, one ML algorithm, and Euclidean distance are good candidates to be used to classify RoQ and Slowloris attacks, but the latter approach with a slightly longer runtime for detecting them.
- Contributions to Permissionless Decentralized Networks for Digital Currencies Based on Delegated Proof of StakePublication . Morais, Rui Pedro Bernardo de; Crocker, Paul Andrew; Sousa, Simão Melo deWith the growing and flourishing of human societies came the desire to exchange what was deemed as valuable, be it a good or a service. Initially this exchange was made directly through barter, either synchronously or asynchronously with debt. The first had the downside of requiring coincidence of wants and the second the need for trust. Both were very inefficient and did not scale well. So, what we call money was invented, which is nothing more than a good that is used as medium of exchange between other goods and services. Since then, money has changed form and has acquired new functions, namely unit of account and store of value. The most recent form of money is digital currency. This money cannot be transferred physically like other forms, so it needs a digital network to be transferred, which can have different characteristics. This thesis concerns a specific type of networks for digital currencies: permissionless, meaning that any participant can have read and write access to the network; decentralized, meaning that no single entity controls the network; and that use Delegated Proof of Stake (DPoS) as a Sybil defence mechanism, to prevent the network from being controlled by malicious actors that create numerous false identities. Its research tries to fulfil the vision that a network for digital currencies, besides being permissionless and decentralized, should be scalable, monetary policy agnostic, anonymous and have high performance. Three different layers of the network are studied: the communication layer, responsible for sending and receiving messages, the transaction layer, responsible for validating those messages, and the consensus layer, responsible for reaching agreement on the state of the network. The first two goals can be achieved in the communication layer. On one hand, a vertical way to scale the system is proposed composed of a peer management and traffic prioritization design based on DPoS, offering an alternative to highly disseminated fee-based models. On the other hand, a horizontal way to scale is presented through database sharding. In the transaction layer, a general framework to make DPoS compatible with anonymity is described. More specifically, two different approaches to achieve amount anonymity are proposed: one based on multi-party computation and the other on the Diffie-Hellman key exchange. Finally, a new decoy selection algorithm, called SimpleDSA, is developed to improve sender anonymity. The consensus layer features two innovative consensus algorithms, Nero and Echidna, and two methods for state machine replication: Sphinx (leader-based) and Cerberus (leaderless). These developments aim to enhance the performance of the network, specifically by decreasing the latency of its state changes and increasing the throughput, i.e., increasing the number of state changes per unit of time. A protocol that instantiates the transaction and consensus layer, called Adamastor, is formalized with security proofs and implemented with a prototype in the Rust language. Benchmarks demonstrate the practicality of the scheme and potential application to decentralized payment systems. While further research is needed, particularly in implementing a fully operational network, it sets a foundation for future advancements. In conclusion, this thesis contributes to the area of knowledge that results from the fusion of economics and computer science, by offering technical solutions for implementing a vision of a more inclusive, fairer, efficient, and secure financial system. The implications of this work are far-reaching, suggesting a future where digital currencies play a significant role in shaping global finance and technology.
- Development of a Salesforce Solution: From Discovery to ImplementationPublication . Martins, Rita Ribeiro; Silva, Frutuoso Gomes Mendes da; Dinkhuysen, Gabriela LevyThis report details the internship experience focused on learning and implementing Salesforce, a prominent platform in the realm of CRM. The initial phase emphasizes an in- -depth immersion into Salesforce, covering essential knowledge assimilation, architectural understanding, and tool familiarization. Concurrently, practical skills in conducting project discoveries are honed, emphasizing the significance of the Discovery phase in a project’s lifecycle. As the internship progresses, the objectives shift towards the hands-on application of Salesforce knowledge and discovery methodologies. The intern is challenged to develop diverse functionalities to fulfill the goals identified during the Discovery phase. This later stage not only requires the application of theoretical knowledge but also demands effective translation of identified requirements into functional and efficient solutions. The overarching objectives of the internship seamlessly intertwine Salesforce proficiency, a deep comprehension of the discovery process, and the skillful application of this knowledge to create functionalities. The report provides a comprehensive overview of the internship journey, capturing the learning curve and practical applications in the dynamic environment of Salesforce and project discovery
- Autonomous emergency braking for highway trajectory planningPublication . Ribeiro, Ricardo André Pereira; Pombo, Nuno Gonçalo Coelho CostaAutonomous vehicles (AV) require critical skills in several scenarios: awareness, intelligent decision-making, and executive control. The improvement of these characteristics is a natural reaction to the emergence of recognition systems such as sensors, which present increasingly precise measurements and a more significant collection of types of data, combined with the emergence of new technologies and mathematical approaches to existing problems in this sector, as well as the evolution in the artificial intelligence sector as raping the transport industry to a new level of automatization. In several areas, all this growth has led to this sector’s rapid development in recent years. The main idea of AVs is to create an intelligent decision-making module capable of controlling all essential processes associated with a vehicle, ranging from creating trajectories or steering control to even risk analysis, as is the case of the Autonomous Emergency Braking (AEB) system. The latter, as ADAS, is carried out mainly to mitigate human errors such as driver distractions, risk analysis with mathematical precision, combat deficiencies in human perception in scenarios with adverse environmental or physical conditions (fatigue, stress, anxiety), and prevent car accidents. This will reduce road traffic, minimize human casualties and injuries, and save millions in monetary loss for all road users. AVs are the future of strengthening and improving security policies in various scenarios. However, their high price in the development and testing process has proven to be a significant deterrent to developing these technologies. Consequently, available solutions in the area of autonomous emergency braking will be addressed, and all new solutions and studies, their strengths and weaknesses, the state of the development process, test systems, quality, and reliability, will be presented. This work aims to create AVs with well-designed trajectory planning using an adaptive Model Predictive Control (MPC) capable of achieving outstanding performance in critical highway scenarios. Furthermore, it also aims to aggregate an emergency braking system that reacts to multidimensional analyses, including collision detection, time to collision (TTC), and braking distance. This study also shows the necessity of placing particular emphasis on verification, validation, and testing (VVT) in the automobile industry, which has contributed significantly to the development of automation systems. They allow developers to test software at a low cost and risk cycle, finding hidden faults in the preliminary phase and increasing confidence in security, functional, and transaction analysis for autonomous prototypes on existing road networks. This is increasingly becoming a norm in the automobile industry thanks to the cost-benefit ratio, allowing the removal of errors before reaching the absolute testing phase, where the cost of mistakes, both monetary and humanitarian, can be catastrophic. For the models to be carried out through simulation environments, this work used Matlab Simulink.
- ML Orchestrator: Development and Optimization of Machine Learning Pipelines and PlatformsPublication . Marques, Pedro Joel da Silva Miroto; Neves, João Carlos Raposo; Lopes, Vasco Ferrinho; Degardin, Bruno ManuelMachine Learning Pipelines play a crucial role in the efficient development of large-scale models, which is a complex process that involves several stages and faces intrinsic challenges. This document seeks to explore the depth of these structures, from the initial preparation of datasets to the final stage of model implementation, as well as the importance of optimizing these structures. Emphasis is also placed on the critical relevance of this process in Cloud Computing environments, where flexibility, scalability and efficiency are imperative. By understanding and properly applying optimized strategies, we not only improve model performance, but also maximize the benefits offered by cloud computing, thus shaping the future of Machine Learning development at scale. The Google Cloud Platform, more specifically the Vertex AI tool, offers a comprehensive solution for building and implementing Machine Learning Pipelines, as it allows development teams to take advantage of pre-trained models, automation of tasks and management of tasks and resources in a simplified way, leading to improved scalability, enabling efficient processing of large volumes of data. In addition, an analysis is made of how the Google Kubernetes Engine tool plays a key role in the management and scaling of these structures, since the ability to manage containers on a large scale guarantees an efficient execution of Machine Learning processes, providing a dynamic response to requests from clients. To efficiently build and optimize a ML pipeline, essential objectives were set to ensure robustness and efficiency. This includes creating a Google Kubernetes Cluster with its complementary services in GKE for the Playground Tool service, employing scalability strategies like KEDA and deploying the DeepNeuronicML model for objects and actions predictions from real-time video streams. Additionally, a Copilot is used to monitor computational resources, ensuring the ML pipeline can manage multiple clients and their AI models in an optimized and scalable manner. To conclude, it’s important to note that optimizing Machine Learning Pipelines in cloud environments is not just a necessity, but a strategic advantage. By adopting innovative approaches and integrating the tools mentioned above (Vertex AI and Google Kubernetes Engine), business organizations can overcome the complex challenges of these structures and boost efficiency and innovation in their Machine Learning services.
- Angular Atomic Components ArchitecturePublication . Pena, Nuno Rodrigo Lopes; Pombo, Nuno Gonçalo Coelho Costa; Aniceto, Alexandre Miguel CoelhoO principal objetivo deste relatório passa por expor todo o trabalho realizado durante o estágio na empresa Emvenci. Este incide na realização de uma reformulação/criação de diversas páginas da aplicação web da empresa utilizando a metodologia de Atomic Design. Através desta reestruturação, um dos principais objetivos é permitir uma melhoria de performance do website, realizando otimizações ao nível do tempo de carregamento de conteúdo, número de linhas de código e tamanho que cada ficheiro ocupa. Desta forma, o documento começa por expor o universo de trabalho da empresa e os seus respetivos objetivos. De seguida, é feita uma descrição da plataforma utilizada pela empresa e uma explicação da metodologia a ser aplicada. No terceiro capítulo, são apresentadas as ferramentas utilizadas, bem como uma explicação das mesmas. Por fim, é retratado o desenvolvimento do estágio, referindo os aspetos mais importantes que foram realizados.
- Processing and Analysis of Positional Time-SeriesPublication . Pedro, Nuno Alexandre Branco Rosa; Fernandes, Rui Manuel da Silva; Crocker, PaulIn the last three decades, GNSS networks have acquired data that allows positional time series of their stations to be calculated. The secular velocity due to the movement of tectonic plates is one of the main parameters to be estimated by processing the collected data, which is normally daily positions of the GNSS stations Hector is a software package that calculates the secular velocity and other associated parameters using such data. Solutions for the parameters of interest are obtained by analyzing the three components separately. Additional parameters can be seasonal variations, noise, and offsets; these parameters can complicate the process of estimating the secular velocity. Therefore, there is a need to use specific software for this calculation. Hector is used on a large scale, meaning the number of time series processed can be large. Therefore, time savings become essential. To optimize this time, changing and combining several parameters is possible. This study suggests some changes that allow, in addition to great time savings, to optimize this software. Running Hector requires the creation of specific configuration files that are sometimes difficult to do by an inexperienced user using commands on a Linux terminal. Therefore, an application was developed to facilitate interaction with Hector. This application has two interaction components with Hector. The first is an API component that allows a user to make requests and receive responses through a command line interface making it easier to create requests in existing environments with a large numbers of stations. The second component is a GUI, where it is also possible to make requests to Hector and obtain the results. All these tools were implemented in the TSA application, developed in PHP, JavaScript and HTML, interacting with Python and TC-Shell scripts that then allow the use of a SQL database together with the application that completes all the functionalities of the application.
- Utilização de técnicas de aprendizagem automática em contexto académico para tipificação do risco de abandono escolarPublication . Pedro, Nkanga; Fazendeiro, Paulo André PaisA evasão de estudantes no ensino superior, é um problema sério e preocupante para as autoridades competentes. Esta questão, não afeta apenas o indivíduo que abandona, mas também a instituição, família e a sociedade em geral. Com o desenvolvimento atual da ciência e da tecnologia, a partir dos dados educacionais registados, a previsão eficiente do abandono dos alunos é atualmente um tema importante de investigação. Este trabalho centra-se na identificação e prevenção do risco de abandono na UNIKIVI com base no desempenho escolar dos alunos. Propomos um conjunto de dados académicos como atributos preditivos e apresentamos modelos de aprendizagem automática que procedem ao agrupamento de alunos utilizando algumas destas características. Espera-se que o modelo sintetizado seja capaz de tipificar o risco de desistência. O objetivo geral é criar um modelo de aprendizagem automática capaz de prever problemas como a evasão dos alunos inscritos e o aumento da taxa de reprovações na instituição. A aplicação desse modelo pode ajudar a identificar os alunos que estão em risco de abandonar a escola, permitindo que a instituição tome medidas preventivas para evitar a evasão escolar. Além disso, a análise dos dados académicos pode ajudar a identificar situações iminentes de abandono e propor ações para minimizar a evasão escolar. Nesse sentido, foram recolhidos dados académicos de cinco cursos diferentes correspondentes a 7 anos letivos(de 2016/2017 a 2022/2023 dos cursos de Contabilidade e gestão, Hidráulica e saneamento de água, Agronomia, Engenharia Informática e Enfermagem). Após a recolha do percurso académico dos alunos, a anonimização da informação e o pré-processamento dos dados, foi conduzido um processo de engenharia e seleção de atributos, construindo assim os conjuntos de dados. Para encontrar padrões de comportamento entre os alunos, foi usado um algoritmo de aprendizagem automática de agrupamento, conhecido como modelo Kprototypes. Isso pode ajudar a dividir os alunos em grupos com características semelhantes. Também utilizamos alguns modelos de classificação, como Random Forest (RF), Support Vector Machines (SVM) e Decision Tree (DT), para prever a probabilidade de desistência de cada aluno com base nesses grupos e, por fim, usamos métricas apropriadas, como precisão, recall e medida F1, para avaliar a precisão dos modelos. A utilização desta metodologia pode ajudar as instituições do ensino superior a melhorar o desempenho dos alunos e reduzir as taxas de evasão escolar.