Showing items 1 - 20 of 69

Your selections:

Discrete-to-deep reinforcement learning methods

- Kurniawan, Budi, Vamplew, Peter, Papasimeon, Michael, Dazeley, Richard, Foale, Cameron

  • Authors: Kurniawan, Budi , Vamplew, Peter , Papasimeon, Michael , Dazeley, Richard , Foale, Cameron
  • Date: 2022
  • Type: Text , Journal article
  • Relation: Neural Computing and Applications Vol. 34, no. 3 (2022), p. 1713-1733
  • Full Text:
  • Reviewed:
  • Description: Neural networks are effective function approximators, but hard to train in the reinforcement learning (RL) context mainly because samples are correlated. In complex problems, a neural RL approach is often able to learn a better solution than tabular RL, but generally takes longer. This paper proposes two methods, Discrete-to-Deep Supervised Policy Learning (D2D-SPL) and Discrete-to-Deep Supervised Q-value Learning (D2D-SQL), whose objective is to acquire the generalisability of a neural network at a cost nearer to that of a tabular method. Both methods combine RL and supervised learning (SL) and are based on the idea that a fast-learning tabular method can generate off-policy data to accelerate learning in neural RL. D2D-SPL uses the data to train a classifier which is then used as a controller for the RL problem. D2D-SQL uses the data to initialise a neural network which is then allowed to continue learning using another RL method. We demonstrate the viability of our algorithms with Cartpole, Lunar Lander and an aircraft manoeuvring problem, three continuous-space environments with low-dimensional state variables. Both methods learn at least 38% faster than baseline methods and yield policies that outperform them. © 2021, The Author(s), under exclusive licence to Springer-Verlag London Ltd., part of Springer Nature.

Scalar reward is not enough : a response to Silver, Singh, Precup and Sutton (2021)

- Vamplew, Peter, Smith, Benjamin, Källström, Johan, Ramos, Gabriel, Rădulescu, Roxana, Roijers, Diederik, Hayes, Conor, Heintz, Fredrik, Mannion, Patrick, Libin, Pieter, Dazeley, Richard, Foale, Cameron

  • Authors: Vamplew, Peter , Smith, Benjamin , Källström, Johan , Ramos, Gabriel , Rădulescu, Roxana , Roijers, Diederik , Hayes, Conor , Heintz, Fredrik , Mannion, Patrick , Libin, Pieter , Dazeley, Richard , Foale, Cameron
  • Date: 2022
  • Type: Text , Journal article
  • Relation: Autonomous Agents and Multi-Agent Systems Vol. 36, no. 2 (2022), p.
  • Full Text:
  • Reviewed:
  • Description: The recent paper “Reward is Enough” by Silver, Singh, Precup and Sutton posits that the concept of reward maximisation is sufficient to underpin all intelligence, both natural and artificial, and provides a suitable basis for the creation of artificial general intelligence. We contest the underlying assumption of Silver et al. that such reward can be scalar-valued. In this paper we explain why scalar rewards are insufficient to account for some aspects of both biological and computational intelligence, and argue in favour of explicitly multi-objective models of reward maximisation. Furthermore, we contend that even if scalar reward functions can trigger intelligent behaviour in specific cases, this type of reward is insufficient for the development of human-aligned artificial general intelligence due to unacceptable risks of unsafe or unethical behaviour. © 2022, The Author(s).

The impact of environmental stochasticity on value-based multiobjective reinforcement learning

- Vamplew, Peter, Foale, Cameron, Dazeley, Richard

  • Authors: Vamplew, Peter , Foale, Cameron , Dazeley, Richard
  • Date: 2022
  • Type: Text , Journal article
  • Relation: Neural Computing and Applications Vol. 34, no. 3 (2022), p. 1783-1799
  • Full Text:
  • Reviewed:
  • Description: A common approach to address multiobjective problems using reinforcement learning methods is to extend model-free, value-based algorithms such as Q-learning to use a vector of Q-values in combination with an appropriate action selection mechanism that is often based on scalarisation. Most prior empirical evaluation of these approaches has focused on deterministic environments. This study examines the impact on stochasticity in rewards and state transitions on the behaviour of multi-objective Q-learning. It shows that the nature of the optimal solution depends on these environmental characteristics, and also on whether we desire to maximise the Expected Scalarised Return (ESR) or the Scalarised Expected Return (SER). We also identify a novel aim which may arise in some applications of maximising SER subject to satisfying constraints on the variation in return and show that this may require different solutions than ESR or conventional SER. The analysis of the interaction between environmental stochasticity and multi-objective Q-learning is supported by empirical evaluations on several simple multiobjective Markov Decision Processes with varying characteristics. This includes a demonstration of a novel approach to learning deterministic SER-optimal policies for environments with stochastic rewards. In addition, we report a previously unidentified issue with model-free, value-based approaches to multiobjective reinforcement learning in the context of environments with stochastic state transitions. Having highlighted the limitations of value-based model-free MORL methods, we discuss several alternative methods that may be more suitable for maximising SER in MOMDPs with stochastic transitions. © 2021, The Author(s), under exclusive licence to Springer-Verlag London Ltd., part of Springer Nature.

A prioritized objective actor-critic method for deep reinforcement learning

- Nguyen, Ngoc, Nguyen, Thanh, Vamplew, Peter, Dazeley, Richard, Nahavandi, Saeid

  • Authors: Nguyen, Ngoc , Nguyen, Thanh , Vamplew, Peter , Dazeley, Richard , Nahavandi, Saeid
  • Date: 2021
  • Type: Text , Journal article
  • Relation: Neural Computing and Applications Vol. 33, no. 16 (2021), p. 10335-10349
  • Full Text: false
  • Reviewed:
  • Description: An increasing number of complex problems have naturally posed significant challenges in decision-making theory and reinforcement learning practices. These problems often involve multiple conflicting reward signals that inherently cause agents’ poor exploration in seeking a specific goal. In extreme cases, the agent gets stuck in a sub-optimal solution and starts behaving harmfully. To overcome such obstacles, we introduce two actor-critic deep reinforcement learning methods, namely Multi-Critic Single Policy (MCSP) and Single Critic Multi-Policy (SCMP), which can adjust agent behaviors to efficiently achieve a designated goal by adopting a weighted-sum scalarization of different objective functions. In particular, MCSP creates a human-centric policy that corresponds to a predefined priority weight of different objectives. Whereas, SCMP is capable of generating a mixed policy based on a set of priority weights, i.e., the generated policy uses the knowledge of different policies (each policy corresponds to a priority weight) to dynamically prioritize objectives in real time. We examine our methods by using the Asynchronous Advantage Actor-Critic (A3C) algorithm to utilize the multithreading mechanism for dynamically balancing training intensity of different policies into a single network. Finally, simulation results show that MCSP and SCMP significantly outperform A3C with respect to the mean of total rewards in two complex problems: Food Collector and Seaquest. © 2021, The Author(s), under exclusive licence to Springer-Verlag London Ltd. part of Springer Nature.

An evaluation methodology for interactive reinforcement learning with simulated users

- Bignold, Adam, Cruz, Francisco, Dazeley, Richard, Vamplew, Peter, Foale, Cameron

  • Authors: Bignold, Adam , Cruz, Francisco , Dazeley, Richard , Vamplew, Peter , Foale, Cameron
  • Date: 2021
  • Type: Text , Journal article
  • Relation: Biomimetics Vol. 6, no. 1 (2021), p. 1-15
  • Full Text:
  • Reviewed:
  • Description: Interactive reinforcement learning methods utilise an external information source to evaluate decisions and accelerate learning. Previous work has shown that human advice could significantly improve learning agents’ performance. When evaluating reinforcement learning algorithms, it is common to repeat experiments as parameters are altered or to gain a sufficient sample size. In this regard, to require human interaction every time an experiment is restarted is undesirable, particularly when the expense in doing so can be considerable. Additionally, reusing the same people for the experiment introduces bias, as they will learn the behaviour of the agent and the dynamics of the environment. This paper presents a methodology for evaluating interactive reinforcement learning agents by employing simulated users. Simulated users allow human knowledge, bias, and interaction to be simulated. The use of simulated users allows the development and testing of reinforcement learning agents, and can provide indicative results of agent performance under defined human constraints. While simulated users are no replacement for actual humans, they do offer an affordable and fast alternative for evaluative assisted agents. We introduce a method for performing a preliminary evaluation utilising simulated users to show how performance changes depending on the type of user assisting the agent. Moreover, we describe how human interaction may be simulated, and present an experiment illustrating the applicability of simulating users in evaluating agent performance when assisted by different types of trainers. Experimental results show that the use of this methodology allows for greater insight into the performance of interactive reinforcement learning agents when advised by different users. The use of simulated users with varying characteristics allows for evaluation of the impact of those characteristics on the behaviour of the learning agent. © 2021 by the authors. Licensee MDPI, Basel, Switzerland.

Levels of explainable artificial intelligence for human-aligned conversational explanations

- Dazeley, Richard, Vamplew, Peter, Foale, Cameron, Young, Cameron, Aryal, Sunil, Cruz, Francisco

  • Authors: Dazeley, Richard , Vamplew, Peter , Foale, Cameron , Young, Cameron , Aryal, Sunil , Cruz, Francisco
  • Date: 2021
  • Type: Text , Journal article
  • Relation: Artificial Intelligence Vol. 299, no. (2021), p.
  • Full Text:
  • Reviewed:
  • Description: Over the last few years there has been rapid research growth into eXplainable Artificial Intelligence (XAI) and the closely aligned Interpretable Machine Learning (IML). Drivers for this growth include recent legislative changes and increased investments by industry and governments, along with increased concern from the general public. People are affected by autonomous decisions every day and the public need to understand the decision-making process to accept the outcomes. However, the vast majority of the applications of XAI/IML are focused on providing low-level ‘narrow’ explanations of how an individual decision was reached based on a particular datum. While important, these explanations rarely provide insights into an agent's: beliefs and motivations; hypotheses of other (human, animal or AI) agents' intentions; interpretation of external cultural expectations; or, processes used to generate its own explanation. Yet all of these factors, we propose, are essential to providing the explanatory depth that people require to accept and trust the AI's decision-making. This paper aims to define levels of explanation and describe how they can be integrated to create a human-aligned conversational explanation system. In so doing, this paper will survey current approaches and discuss the integration of different technologies to achieve these levels with Broad eXplainable Artificial Intelligence (Broad-XAI), and thereby move towards high-level ‘strong’ explanations. © 2021 Elsevier B.V.

Potential-based multiobjective reinforcement learning approaches to low-impact agents for AI safety

- Vamplew, Peter, Foale, Cameron, Dazeley, Richard, Bignold, Adam

  • Authors: Vamplew, Peter , Foale, Cameron , Dazeley, Richard , Bignold, Adam
  • Date: 2021
  • Type: Text , Journal article
  • Relation: Engineering Applications of Artificial Intelligence Vol. 100, no. (2021), p.
  • Full Text: false
  • Reviewed:
  • Description: The concept of impact-minimisation has previously been proposed as an approach to addressing the safety concerns that can arise from utility-maximising agents. An impact-minimising agent takes into account the potential impact of its actions on the state of the environment when selecting actions, so as to avoid unacceptable side-effects. This paper proposes and empirically evaluates an implementation of impact-minimisation within the framework of multiobjective reinforcement learning. The key contributions are a novel potential-based approach to specifying a measure of impact, and an examination of a variety of non-linear action-selection operators so as to achieve an acceptable trade-off between achieving the agent's primary task and minimising environmental impact. These experiments also highlight a previously unreported issue with noisy estimates for multiobjective agents using non-linear action-selection, which has broader implications for the application of multiobjective reinforcement learning. © 2021

Reanimating historic malware samples

- Black, Paul, Gondal, Iqbal, Vamplew, Peter, Lakhotia, Arun

  • Authors: Black, Paul , Gondal, Iqbal , Vamplew, Peter , Lakhotia, Arun
  • Date: 2021
  • Type: Text , Book chapter
  • Relation: Malware Analysis Using Artificial Intelligence and Deep Learning p. 345-360
  • Full Text: false
  • Reviewed:
  • Description: Many types of malicious software are controlled from an attacker’s command and control (C2) servers. Anti-virus organizations seek to defeat malware attacks by requesting removal of C2 server Domain Name Server (DNS) records. As a result, the life span of most malware samples is relatively short. Large datasets of historical malware samples are available for countermeasures research. However, due to the age of these malware samples, their C2 servers are no longer available. To cope with high volumes of malware production, malware analysis is increasingly performed using machine learning techniques. Dynamic analysis is commonly used for feature extraction. However, due to the absence of their C2 servers, after initialization, malware samples may exit or loop attempting to establish C2 server connections and, as a result, no longer exhibit their original capabilities. Therefore, partial execution of historical malware samples in a sandbox results in features that differ from those that would be extracted in-the-wild, thus invalidating the results of any machine learning research based on these features. One approach to extracting accurate features is to build an emulated C2 server to provide an environment that allows control of the full capabilities of the malware in an isolated environment. To illustrate the benefits of building C2 server emulators, this chapter provides examples of techniques for the creation of C2 server emulators for three malware families (Zeus, CryptoWall, and CryptoLocker) using manual reverse engineering techniques and a review of semi-automated techniques for the construction of C2 server emulators.

A multi-objective deep reinforcement learning framework

- Nguyen, Thanh, Nguyen, Ngoc, Vamplew, Peter, Nahavandi, Saeid, Dazeley, Richard, Lim, Chee

  • Authors: Nguyen, Thanh , Nguyen, Ngoc , Vamplew, Peter , Nahavandi, Saeid , Dazeley, Richard , Lim, Chee
  • Date: 2020
  • Type: Text , Journal article
  • Relation: Engineering Applications of Artificial Intelligence Vol. 96, no. (2020), p.
  • Full Text:
  • Reviewed:
  • Description: This paper introduces a new scalable multi-objective deep reinforcement learning (MODRL) framework based on deep Q-networks. We develop a high-performance MODRL framework that supports both single-policy and multi-policy strategies, as well as both linear and non-linear approaches to action selection. The experimental results on two benchmark problems (two-objective deep sea treasure environment and three-objective Mountain Car problem) indicate that the proposed framework is able to find the Pareto-optimal solutions effectively. The proposed framework is generic and highly modularized, which allows the integration of different deep reinforcement learning algorithms in different complex problem domains. This therefore overcomes many disadvantages involved with standard multi-objective reinforcement learning methods in the current literature. The proposed framework acts as a testbed platform that accelerates the development of MODRL for solving increasingly complicated multi-objective problems. © 2020 Elsevier Ltd

API based discrimination of ransomware and benign cryptographic programs

- Black, Paul, Sohail, Ammar, Gondal, Iqbal, Kamruzzaman, Joarder, Vamplew, Peter, Watters, Paul

Function similarity using family context

- Black, Paul, Gondal, Iqbal, Vamplew, Peter, Lakhotia, Arun

  • Authors: Black, Paul , Gondal, Iqbal , Vamplew, Peter , Lakhotia, Arun
  • Date: 2020
  • Type: Text , Journal article
  • Relation: Electronics Vol. 9, no. 7 (Jul 2020), p. 20
  • Full Text:
  • Reviewed:
  • Description: Finding changed and similar functions between a pair of binaries is an important problem in malware attribution and for the identification of new malware capabilities. This paper presents a new technique called Function Similarity using Family Context (FSFC) for this problem. FSFC trains a Support Vector Machine (SVM) model using pairs of similar functions from two program variants. This method improves upon previous research called Cross Version Contextual Function Similarity (CVCFS) e epresenting a function using features extracted not just from the function itself, but also, from other functions with which it has a caller and callee relationship. We present the results of an initial experiment that shows that the use of additional features from the context of a function significantly decreases the false positive rate, obviating the need for a separate pass for cleaning false positives. The more surprising and unexpected finding is that the SVM model produced by FSFC can abstract function similarity features from one pair of program variants to find similar functions in an unrelated pair of program variants. If validated by a larger study, this new property leads to the possibility of creating generic similar function classifiers that can be packaged and distributed in reverse engineering tools such as IDA Pro and Ghidra.
  • Description: This research was performed in the Internet Commerce Security Lab (ICSL), which is a joint venture with research partners Westpac, IBM, and Federation University Australia.

Hybrid intrusion detection system based on the stacking ensemble of C5 decision tree classifier and one class support vector machine

- Khraisat, Ansam, Gondal, Iqbal, Vamplew, Peter, Kamruzzaman, Joarder, Alazab, Ammar

  • Authors: Khraisat, Ansam , Gondal, Iqbal , Vamplew, Peter , Kamruzzaman, Joarder , Alazab, Ammar
  • Date: 2020
  • Type: Text , Journal article
  • Relation: Electronics (Switzerland) Vol. 9, no. 1 (2020), p.
  • Full Text:
  • Reviewed:
  • Description: Cyberttacks are becoming increasingly sophisticated, necessitating the efficient intrusion detection mechanisms to monitor computer resources and generate reports on anomalous or suspicious activities. Many Intrusion Detection Systems (IDSs) use a single classifier for identifying intrusions. Single classifier IDSs are unable to achieve high accuracy and low false alarm rates due to polymorphic, metamorphic, and zero-day behaviors of malware. In this paper, a Hybrid IDS (HIDS) is proposed by combining the C5 decision tree classifier and One Class Support Vector Machine (OC-SVM). HIDS combines the strengths of SIDS) and Anomaly-based Intrusion Detection System (AIDS). The SIDS was developed based on the C5.0 Decision tree classifier and AIDS was developed based on the one-class Support Vector Machine (SVM). This framework aims to identify both the well-known intrusions and zero-day attacks with high detection accuracy and low false-alarm rates. The proposed HIDS is evaluated using the benchmark datasets, namely, Network Security Laboratory-Knowledge Discovery in Databases (NSL-KDD) and Australian Defence Force Academy (ADFA) datasets. Studies show that the performance of HIDS is enhanced, compared to SIDS and AIDS in terms of detection rate and low false-alarm rates. © 2020 by the authors. Licensee MDPI, Basel, Switzerland.

Identifying cross-version function similarity using contextual features

- Black, Paul, Gondal, Iqbal, Vamplew, Peter, Lakhotia, Arun

  • Authors: Black, Paul , Gondal, Iqbal , Vamplew, Peter , Lakhotia, Arun
  • Date: 2020
  • Type: Text , Conference paper
  • Relation: 19th IEEE International Conference on Trust, Security and Privacy in Computing and Communications, TrustCom 2020 p. 810-818
  • Full Text: false
  • Reviewed:
  • Description: The identification of similar functions in malware assists analysis by supporting the exclusion of functions that have been previously analysed, allows the identification of new variants, supports authorship attribution, and the analysis of malware phylogeny. A function's context is a set comprising the function itself and all the program functions that may be executed when this function is called. Contextual features consist of data that is extracted from the functions contained in the function context. This paper presents a novel technique called Cross Version Contextual Function Similarity (CVCFS) to identify function pairs in two programs using features based on both individual functions and function context. The CVCFS technique uses Support Vector Machine (SVM) machine learning of function similarity features to pre-filter function pairs and then applies an edit distance technique using function semantics to reduce false positives. A case study is provided where individual and contextual features are extracted from three versions of Zeus malware. The SVM pre-filtering, followed by the use of an edit distance technique to filter false positives, gives a function pair identification accuracy of 85 percent. © 2020 IEEE.

Motivational factors of Australian mobile gamers

- Greenwood, Jordan, Achterbosch, Leigh, Meredith, Grant, Vamplew, Peter

  • Authors: Greenwood, Jordan , Achterbosch, Leigh , Meredith, Grant , Vamplew, Peter
  • Date: 2020
  • Type: Text , Conference proceedings , Conference paper
  • Relation: Proceedings of the Australasian Computer Science Week Multiconference (ACSW 2020); Melbourne, Australia; 4th-6th February 2020 p. 6
  • Full Text:
  • Reviewed:
  • Description: Mobile games are a fast growing industry, overtaking all other video game platforms with year on year increases in revenue. Many studies have been conducted to explore the motivations of why video games players play their selected games. However very little research has focused on mobile gamers. In addition, Australian studies on the topic are sparse. This paper aimed to discover what motivates a mobile gamer from the perspective of the initial motivational factors attracting them to a mobile game, and the motivational factors that provide interest to continue playing and thereby increase game longevity. A survey was conducted online for Australian participants, which attracted 123 respondents. The survey was formulated by focusing on the 12 key subcomponents as motivational factors of the Gamer Motivational Profile v2 model devised by Quantic Foundry. It was discovered that mobile gamers are a completely different breed of gamer in contrast to the general video gamer. Strategy and challenge which are subcomponents of mastery proved popular among all mobile gamers, while destruction and excitement, subcomponents of action, were often the least motivating factors of all. With the newly discovered data, perhaps mobile game developers can pursue the correct avenues of game design when catering to their target audience.

A novel ensemble of hybrid intrusion detection system for detecting internet of things attacks

- Khraisat, Ansam, Gondal, Iqbal, Vamplew, Peter, Kamruzzaman, Joarder, Alazab, Ammar

  • Authors: Khraisat, Ansam , Gondal, Iqbal , Vamplew, Peter , Kamruzzaman, Joarder , Alazab, Ammar
  • Date: 2019
  • Type: Text , Journal article
  • Relation: Electronics (Switzerland) Vol. 8, no. 11 (2019), p.
  • Full Text:
  • Reviewed:
  • Description: The Internet of Things (IoT) has been rapidly evolving towards making a greater impact on everyday life to large industrial systems. Unfortunately, this has attracted the attention of cybercriminals who made IoT a target of malicious activities, opening the door to a possible attack to the end nodes. Due to the large number and diverse types of IoT devices, it is a challenging task to protect the IoT infrastructure using a traditional intrusion detection system. To protect IoT devices, a novel ensemble Hybrid Intrusion Detection System (HIDS) is proposed by combining a C5 classifier and One Class Support Vector Machine classifier. HIDS combines the advantages of Signature Intrusion Detection System (SIDS) and Anomaly-based Intrusion Detection System (AIDS). The aim of this framework is to detect both the well-known intrusions and zero-day attacks with high detection accuracy and low false-alarm rates. The proposed HIDS is evaluated using the Bot-IoT dataset, which includes legitimate IoT network traffic and several types of attacks. Experiments show that the proposed hybrid IDS provide higher detection rate and lower false positive rate compared to the SIDS and AIDS techniques. © 2019 by the authors. Licensee MDPI, Basel, Switzerland.

Categorical features transformation with compact one-hot encoder for fraud detection in distributed environment

- Ul Haq, Ikram, Gondal, Iqbal, Vamplew, Peter, Brown, Simon

  • Authors: Ul Haq, Ikram , Gondal, Iqbal , Vamplew, Peter , Brown, Simon
  • Date: 2019
  • Type: Text , Conference proceedings , Conference paper
  • Relation: 2019 16th Australasian Conference on Data Mining, AusDM 2018; Bathurst, NSW; 28 November 2018 through 30 November 2018 Vol. 996, p. 69-80
  • Full Text: false
  • Reviewed:
  • Description: Fraud detection for online banking is an important research area, but one of the challenges is the heterogeneous nature of transactions data i.e. a combination of numeric as well as mixed attributes. Usually, numeric format data gives better performance for classification, regression and clustering algorithms. However, many machine learning problems have categorical, or nominal features, rather than numeric features only. In addition, some machine learning platforms such as Apache Spark accept numeric data only. One-hot Encoding (OHE) is a widely used approach for transforming categorical features to numerical features in traditional data mining tasks. The one-hot approach has some challenges as well: the sparseness of the transformed data and that the distinct values of an attribute are not always known in advance. Other than the model accuracy, compactness of machine learning models is equally important due to growing memory and storage needs. This paper presents an innovative technique to transform categorical features to numeric features by compacting sparse data even if all the distinct values are not known. The transformed data can be used for the development of fraud detection systems. The accuracy of the results has been validated on synthetic and real bank fraud data and a publicly available anomaly detection (KDD-99) dataset on a multi-node data cluster. © Springer Nature Singapore Pte Ltd. 2019.

Evolved similarity techniques in malware analysis

- Black, Paul, Gondal, Iqbal, Vamplew, Peter, Lakhotia, Arun

  • Authors: Black, Paul , Gondal, Iqbal , Vamplew, Peter , Lakhotia, Arun
  • Date: 2019
  • Type: Text , Conference proceedings
  • Relation: 2019 18th IEEE International Conference On Trust, Security And Privacy; published in In Computing And Communications/13th IEEE International Conference On Big Data Science And Engineering (TrustCom/BigDataSE), 5-8th Aug, 2019 p. 404-410
  • Full Text: false
  • Reviewed:
  • Description: Malware authors are known to reuse existing code, this development process results in software evolution and a sequence of versions of a malware family containing functions that show a divergence from the initial version. This paper proposes the term evolved similarity to account for this gradual divergence of similarity across the version history of a malware family. While existing techniques are able to match functions in different versions of malware, these techniques work best when the version changes are relatively small. This paper introduces the concept of evolved similarity and presents automated Evolved Similarity Techniques (EST). EST differs from existing malware function similarity techniques by focusing on the identification of significantly modified functions in adjacent malware versions and may also be used to identify function similarity in malware samples that differ by several versions. The challenge in identifying evolved malware function pairs lies in identifying features that are relatively invariant across evolved code. The research in this paper makes use of the function call graph to establish these features and then demonstrates the use of these techniques using Zeus malware.

Integrating biological heuristics and gene expression data for gene regulatory network inference

- Zarnegar, Armita, Jelinek, Herbert, Vamplew, Peter, Stranieri, Andrew

  • Authors: Zarnegar, Armita , Jelinek, Herbert , Vamplew, Peter , Stranieri, Andrew
  • Date: 2019
  • Type: Text , Conference proceedings , Conference paper
  • Relation: 2019 Australasian Computer Science Week Multiconference, ACSW 2019; Sydney, Australia; 29th-31st January 2019 p. 1-10
  • Full Text: false
  • Reviewed:
  • Description: Gene Regulatory Networks (GRNs) offer enhanced insight into the biological functions and biochemical pathways of cells associated with gene regulatory mechanisms. However, obtaining accurate GRNs that explain gene expressions and functional associations remains a difficult task. Only a few studies have incorporated heuristics into a GRN discovery process. Doing so has the potential to improve accuracy and reduce the search space and computational time. A technique for GRN discovery that integrates heuristic information into the discovery process is advanced. The approach incorporates three elements: 1) a novel 2D visualized coexpression function that measures the association between genes; 2) a post-processing step that improves detection of up, down and self-regulation and 3) the application of heuristics to generate a Hub network as the backbone of the GRN. Using available microarray and next generation sequencing data from Escherichia coli, six synthetic benchmark GRN datasets were generated with the neighborhood addition and cluster addition methods available in SynTReN. Results of the novel 2D-visualization co-expression function were compared with results obtained using Pearson's correlation and mutual information. The performance of the biological genetics-based heuristics consisting of the 2D-Visualized Co-expression function, post-processing and Hub network was then evaluated by comparing the performance to the GRNs obtained by ARACNe and CLR. The 2D-Visualized Co-expression function significantly improved gene-gene association matching compared to Pearson's correlation coefficient (t = 3.46, df = 5, p = 0.02) and Mutual Information (t = 4.42, df = 5, p = 0.007). The heuristics model gave a 60% improvement against ARACNe (p = 0.02) and CLR (p = 0.019). Analysis of Escherichia coli data suggests that the GRN discovery technique proposed is capable of identifying significant transcriptional regulatory interactions and the corresponding regulatory networks.

Memory-Based explainable reinforcement learning

- Cruz, Francisco, Dazeley, Richard, Vamplew, Peter

  • Authors: Cruz, Francisco , Dazeley, Richard , Vamplew, Peter
  • Date: 2019
  • Type: Text , Conference proceedings
  • Relation: 32nd Australasian Joint Conference on Artificial Intelligence AI , Switzerland, 2019 in AI 2019 ; Advances in artificial intelligence ; published in Lecture Notes in Computer Science, vol 11919 , pp.66-77
  • Full Text: false
  • Description: Reinforcement learning (RL) is a learning approach based on behavioral psychology used by artificial agents to learn autonomously by interacting with their environment. An open issue in RL is the lack of visibility and understanding for end-users in terms of decisions taken by an agent during the learning process. One way to overcome this issue is to endow the agent with the ability to explain in simple terms why a particular action is taken in a particular situation. In this work, we propose a memory-based explainable reinforcement learning (MXRL) approach. Using an episodic memory, the RL agent is able to explain its decisions by using the probability of success and the number of transactions to reach the goal state. We have performed experiments considering two variations of a simulated scenario, namely, an unbounded grid world with aversive regions and a bounded grid world. The obtained results show that the agent, using information extracted from the memory, is able to explain its behavior in an understandable manner for non-expert end-users at any moment during its operation.

Memory-Based explainable reinforcement learning

- Cruz, Francisco, Dazeley, Richard, Vamplew, Peter

  • Authors: Cruz, Francisco , Dazeley, Richard , Vamplew, Peter
  • Date: 2019
  • Type: Text , Conference proceedings
  • Relation: 32nd Australasian Joint Conference on Artificial Intelligence AI , Switzerland, 2019 in AI 2019 ; Advances in artificial intelligence ; published in Lecture Notes in Computer Science, vol 11919 , pp.66-77
  • Full Text: false
  • Reviewed:
  • Description: Reinforcement learning (RL) is a learning approach based on behavioral psychology used by artificial agents to learn autonomously by interacting with their environment. An open issue in RL is the lack of visibility and understanding for end-users in terms of decisions taken by an agent during the learning process. One way to overcome this issue is to endow the agent with the ability to explain in simple terms why a particular action is taken in a particular situation. In this work, we propose a memory-based explainable reinforcement learning (MXRL) approach. Using an episodic memory, the RL agent is able to explain its decisions by using the probability of success and the number of transactions to reach the goal state. We have performed experiments considering two variations of a simulated scenario, namely, an unbounded grid world with aversive regions and a bounded grid world. The obtained results show that the agent, using information extracted from the memory, is able to explain its behavior in an understandable manner for non-expert end-users at any moment during its operation.