Showing items 1 - 20 of 75

Your selections:

Pre-trained language models with limited data for intent classification

- Kasthuriarachchy, Buddhika, Chetty, Madhu, Karmakar, Gour, Walls, Darren

  • Authors: Kasthuriarachchy, Buddhika , Chetty, Madhu , Karmakar, Gour , Walls, Darren
  • Date: 2020
  • Type: Text , Conference proceedings , Conference paper
  • Relation: 2020 International Joint Conference on Neural Networks, IJCNN 2020
  • Full Text: false
  • Reviewed:
  • Description: Intent analysis is capturing the attention of both the industry and academia due to its commercial and noncommercial significance. The rapid growth of unstructured data of micro-blogging platforms, such as Twitter and Facebook, are amongst the important sources for intent analysis. However, the social media data are often noisy and diverse, thus making the task very challenging. Further, the intent analysis frequently suffers from lack of sufficient data because the labeled datasets are often manually annotated. Recently, BERT (Bidirectional Encoder Representation from Transformers), a state-of-the-art language representation model, has attracted attention for accurate language modelling. In this paper, we investigate the application of BERT for its suitability for intent analysis. We study the fine-tuning of the BERT model through inductive transfer learning and investigate methods to overcome the challenges due to limited data availability by proposing a novel semantic data augmentation approach. This technique generates synthetic sentences while preserving the label-compatibility using the semantic meaning of the sentences, to improve the intent classification accuracy. Thus, based on the considerations for finetuning and data augmentation, a systematic and novel step-bystep methodology is presented for applying the linguistic model BERT for intent classification with limited data available. Our results show that the pre-trained language can be effectively used with noisy social media data to achieve state-of-the-art accuracy in intent analysis under low labeled-data regime. Moreover, our results also confirm that the proposed text augmentation technique is effective in eliminating noisy synthetic sentences, thereby achieving further performance improvements. © 2020 IEEE.

mDBN: motif based learning of gene regulatory networks using dynamic Bayesian networks

- Morshed, Nizamul, Chetty, Madhu, Nguyen, Vinh, Caelli, Terry

Combining segmental semi-Markov models with neural networks for protein secondary structure prediction

- Bidargaddi, Niranjan, Chetty, Madhu, Kamruzzaman, Joarder

  • Authors: Bidargaddi, Niranjan , Chetty, Madhu , Kamruzzaman, Joarder
  • Date: 2009
  • Type: Text , Journal article
  • Relation: Neurocomputing Vol. 72, no. 3943-3950 (2009), p.3943-3950
  • Full Text: false
  • Reviewed:
  • Description: Predicting the secondary structure of proteins from a primary sequence alone has been variously approached from either a classification or a generative model perspective. The most prominent classification methods have used neural networks, which involves mappings from a local window of residues in the sequence to the structural state of the central residue in the window, thus capturing the local interactions effectively. However, they fail to capture distant interactions among residues. The generative models based on Bayesian segmentation capture sequence structure relationships using generalized hidden Markov models with explicit state duration. They capture non-local interactions through a joint sequence-structure probability distribution based on structural segments. In this paper, we investigate a combined architecture of Bayesian segmentation at the first stage and neural network at the second stage which captures both local and non-local correlation, to increase the single sequence prediction accuracy. The combined architecture is further enhanced by using neural network optimization and ensemble techniques.

Clustered memetic algorithm for protein structure prediction

- Islam, M. D., Chetty, Madhu

Data discretization for dynamic Bayesian network based modeling of genetic networks

- Nguyen, Vinh, Chetty, Madhu, Coppel, Ross, Wangikar, Pramod

  • Authors: Nguyen, Vinh , Chetty, Madhu , Coppel, Ross , Wangikar, Pramod
  • Date: 2012
  • Type: Text , Conference paper
  • Relation: Neural Information Processing 19th International Conference p. 298-306
  • Full Text: false
  • Reviewed:
  • Description: Dynamic Bayesian networks (DBN) are widely applied in Systems biology for modeling various biological networks, including gene regulatory networks and metabolic networks. The application of DBN models often requires data discretization. Although various discretization techniques exist, currently there is no consensus on which approach is most suitable. Popular discretization strategies within the bioinformatics community, such as interval and quantile discretization, are likely not optimal. In this paper, we propose a novel approach for data discretization for mutual information based learning of DBN. In this approach, the data are discretized so that the mutual information between parent and child nodes is maximized, subject to a suitable penalty put on the complexity of the discretization. A dynamic programming approach is used to find the optimal discretization threshold for each individual variable. Our approach iteratively learns both the network and the discretization scheme until a locally optimal solution is reached. Tests on real genetic networks confirm the effectiveness of the proposed method.

Rhythmic and sustained oscillations in metabolism and gene expression of Cyanothece sp. ATCC 51142 under constant light

- Gaudana, Sandeep, Krishnakumar, S., Alagesan, Swathi, Digmurti, Madhuri, Viswanathan, Ganesh, Chetty, Madhu, Wangikar, Pramod

  • Authors: Gaudana, Sandeep , Krishnakumar, S. , Alagesan, Swathi , Digmurti, Madhuri , Viswanathan, Ganesh , Chetty, Madhu , Wangikar, Pramod
  • Date: 2013
  • Type: Text , Journal article
  • Relation: Frontiers in Microbiology Vol. 4, no. Article 374 (2013), p. 1-11
  • Full Text:
  • Reviewed:
  • Description: Cyanobacteria, a group of photosynthetic prokaryotes, oscillate between day and night time metabolisms with concomitant oscillations in gene expression in response to light/dark cycles (LD). The oscillations in gene expression have been shown to sustain in constant light (LL) with a free running period of 24 h in a model cyanobacterium Synechococcus elongatus PCC 7942. However, equivalent oscillations in metabolism are not reported under LL in this non-nitrogen fixing cyanobacterium. Here we focus on Cyanothece sp. ATCC 51142, a unicellular, nitrogen-fixing cyanobacterium known to temporally separate the processes of oxygenic photosynthesis and oxygen-sensitive nitrogen fixation. In a recent report, metabolism of Cyanothece 51142 has been shown to oscillate between photosynthetic and respiratory phases under LL with free running periods that are temperature dependent but significantly shorter than the circadian period. Further, the oscillations shift to circadian pattern at moderate cell densities that are concomitant with slower growth rates. Here we take this understanding forward and demonstrate that the ultradian rhythm under LL sustains at much higher cell densities when grown under turbulent regimes that simulate flashing light effect. Our results suggest that the ultradian rhythm in metabolism may be needed to support higher carbon and nitrogen requirements of rapidly growing cells under LL. With a comprehensive Real time PCR based gene expression analysis we account for key regulatory interactions and demonstrate the interplay between clock genes and the genes of key metabolic pathways. Further, we observe that several genes that peak at dusk in Synechococcus peak at dawn in Cyanothece and vice versa. The circadian rhythm of this organism appears to be more robust with peaking of genes in anticipation of the ensuing photosynthetic and respiratory metabolic phases.

Binary-organoid particle swarm optimisation for inferring genetic networks

- Chanthaphavong, Santi, Chetty, Madhu

  • Authors: Chanthaphavong, Santi , Chetty, Madhu
  • Date: 2010
  • Type: Text , Conference paper
  • Relation: Evolutionary Computation (CEC), 2010 IEEE Congress
  • Full Text: false
  • Reviewed:
  • Description: A holistic understanding of genetic interactions is crucial in the analysis of complex biological systems. However, due to the dimensionality problem (less samples and large number of genes) of microarray data, obtaining an optimal gene regulatory network is not only difficult but also computationally expensive. In this paper, a Bayesian model for the genetic interactions using the Minimum Description Length as a scoring metric is proposed. For fast optimisation of the network structure, we propose a novel Swarm Intelligence algorithm called Binary-Organoid Particle Swarm (BORG-Swarm). In BORG-Swarm we introduce the concepts of probability threshold vector and particle drift to update particle positions. Experimental studies are carried out using real-life yeast cell cycle dataset. Results indicate that existing binary swarms fail to converge and suffer from long runtimes. In constrast, BORG-Swarm's fast convergence towards the global optimum becomes apparent from results of extensive simulations.

Extended HP model for protein structure prediction

- Hoque, Md Tamjidul, Chetty, Madhu, Sattar, Abdul

  • Authors: Hoque, Md Tamjidul , Chetty, Madhu , Sattar, Abdul
  • Date: 2009
  • Type: Text , Journal article
  • Relation: Computational Biology and Bioinformatics Vol. Jan-Feb 2011, no. (2009 ), p. 234-245
  • Full Text: false
  • Reviewed:
  • Description: This paper presents the impact of twins and the measures for their removal from the population of genetic algorithm (GA) when applied to effective conformational searching. It is conclusively shown that a twin removal strategy for a GA provides considerably enhanced performance when investigating solutions to complex ab initio protein structure prediction (PSP) problems in low-resolution model. Without twin removal, GA crossover and mutation operations can become ineffectual as generations lose their ability to produce significant differences, which can lead to the solution stalling. The paper relaxes the definition of chromosomal twins in the removal strategy to not only encompass identical, but also highly correlated chromosomes within the GA population, with empirical results consistently exhibiting significant improvements solving PSP problems.

GlobalMIT: learning globally optimal dynamic Bayesian network with the mutual information test criterion

- Nguyen, Vinh, Chetty, Madhu, Coppel, Ross, Wangikar, Pramod

MCMC based Bayesian inference for modelling gene networks

- Ram, Ramesh, Chetty, Madhu

  • Authors: Ram, Ramesh , Chetty, Madhu
  • Date: 2009
  • Type: Text , Conference paper
  • Relation: Pattern Recognition in Bioinformatics p. 293-306
  • Full Text: false
  • Reviewed:
  • Description: In this paper, we apply Bayesian networks (BN) to infer gene regulatory network (GRN) model from gene expression data. This inference process, consisting of structure search and conditional probability estimation, is challenging due to the size and quality of the data that is currently available. Our previous studies for GRN reconstruction involving evolutionary search algorithm obtained a most plausible graph structure referred as Independence-map (or simply I-map). However, the limitations of the data (large number of genes and less samples) can result in many plausible structures that equally satisfy the data set. In the present study, given the network structures, we estimate the conditional probability distribution of each variable (gene) from the data set to deduce a unique minimal I-map. This is achieved by using Markov Chain Monte Carlo (MCMC) method whereby the search space is iteratively reduced resulting in the required convergence within a reasonable computation time. We present empirical results on both, the synthetic and real-life data sets and also compare our approach with the plain MCMC sampling approach. The inferred minimal I-map on the real-life yeast data set is also presented.

Local and global algorithms for learning dynamic Bayesian networks

- Nguyen, Vinh, Chetty, Madhu, Coppel, Ross, Wangikar, Pramod

  • Authors: Nguyen, Vinh , Chetty, Madhu , Coppel, Ross , Wangikar, Pramod
  • Date: 2012
  • Type: Text , Conference paper
  • Relation: The 12th IEEE International Conference on Data Mining (ICDM 2012) p. 685-694
  • Full Text: false
  • Reviewed:
  • Description: Learning optimal Bayesian networks (BN) from data is NP-hard in general. Nevertheless, certain BN classes with additional topological constraints, such as the dynamic BN (DBN) models, widely applied in specific fields such as systems biology, can be efficiently learned in polynomial time. Such algorithms have been developed for the Bayesian-Dirichlet (BD), Minimum Description Length (MDL), and Mutual Information Test (MIT) scoring metrics. The BD-based algorithm admits a large polynomial bound, hence it is impractical for even modestly sized networks. The MDL-and MIT-based algorithms admit much smaller bounds, but require a very restrictive assumption that all variables have the same cardinality, thus significantly limiting their applicability. In this paper, we first propose an improvement to the MDL-and MIT-based algorithms, dropping the equicardinality constraint, thus significantly enhancing their generality. We also explore local Markov blanket based algorithms for constructing BN in the context of DBN, and show an interesting result: under the faithfulness assumption, the mutual information test based local Markov blanket algorithms yield the same network as learned by the global optimization MIT-based algorithm. Experimental validation on small and large scale genetic networks demonstrates the effectiveness of our proposed approaches.

Generating synthetic gene regulatory networks

- Ram, Ramesh, Chetty, Madhu

  • Authors: Ram, Ramesh , Chetty, Madhu
  • Date: 2008
  • Type: Text , Conference paper
  • Relation: Third IAPR International Conference, PRIB
  • Full Text: false
  • Reviewed:
  • Description: Reconstructing GRN from microarray dataset is a very challenging problem as these datasets typically have large number of genes and less number of samples. Moreover, the reconstruction task becomes further complicated as there are no suitable synthetic datasets available for validation and evaluation of GRN reconstruction techniques. Synthetic datasets allow validating new techniques and approaches since the underlying mechanisms of the GRNs, generated from these datasets, are completely known. In this paper, we present an approach for synthetically generating gene networks using causal relationships. The synthetic networks can have varying topologies such as small world, random, scale free, or hierarchical topologies based on the well-defined GRN properties. These artificial but realistic GRN networks provide a simulation environment similar to a real-life laboratory microarray experiment. These networks also provide a mechanism for studying the robustness of reconstruction methods to individual and combination of parametric changes such as topology, noise (background and experimental noise) and time delays. Studies involving complicated interactions such as feedback loops, oscillations, bi-stability, dynamic behavior, vertex in-degree changes and number of samples can also be carried out by the proposed synthetic GRN networks.

Gene regulatory network modeling via global optimization of high-order dynamic Bayesian network

- Nguyen, Vinh, Chetty, Madhu, Coppel, Ross, Wangikar, Pramod

  • Authors: Nguyen, Vinh , Chetty, Madhu , Coppel, Ross , Wangikar, Pramod
  • Date: 2012
  • Type: Text , Journal article
  • Relation: BMC Bioinformatics Vol. 13, no. 131 (2012), p. 1-16
  • Full Text:
  • Reviewed:
  • Description: Abstract Background Dynamic Bayesian network (DBN) is among the mainstream approaches for modeling various biological networks, including the gene regulatory network (GRN). Most current methods for learning DBN employ either local search such as hill-climbing, or a meta stochastic global optimization framework such as genetic algorithm or simulated annealing, which are only able to locate sub-optimal solutions. Further, current DBN applications have essentially been limited to small sized networks. Results To overcome the above difficulties, we introduce here a deterministic global optimization based DBN approach for reverse engineering genetic networks from time course gene expression data. For such DBN models that consist only of inter time slice arcs, we show that there exists a polynomial time algorithm for learning the globally optimal network structure. The proposed approach, named GlobalMIT+, employs the recently proposed information theoretic scoring metric named mutual information test (MIT). GlobalMIT+ is able to learn high-order time delayed genetic interactions, which are common to most biological systems. Evaluation of the approach using both synthetic and real data sets, including a 733 cyanobacterial gene expression data set, shows significantly improved performance over other techniques. Conclusions Our studies demonstrate that deterministic global optimization approaches can infer large scale genetic networks.

Clustered memetic algorithm with local heuristics for ab initio protein structure prediction

- Islam, M. D., Chetty, Madhu

  • Authors: Islam, M. D. , Chetty, Madhu
  • Date: 2013
  • Type: Text , Journal article
  • Relation: IEEE Transactions on Evolutionary Computation Vol. 17, no. 4 (2013), p. 558-576
  • Full Text: false
  • Reviewed:
  • Description: Low-resolution protein models are often used within a hierarchical framework for structure prediction. However, even with these simplified but realistic protein models, the search for the optimal solution remains NP complete. The complexity is further compounded by the multimodal nature of the search space. In this paper, we propose a systematic design of an evolutionary search technique, namely the memetic algorithm (MA), to effectively search the vast search space by exploiting the domain-specific knowledge and taking cognizance of the multimodal nature of the search space. The proposed MA achieves this by incorporating various novel features: 1) a modified fitness function includes two additional terms to account for the hydrophobic and polar nature of the residues; 2) a systematic (rather than random) generation of population automatically prevents an occurrence of invalid conformations; 3) a generalized nonisomorphic encoding scheme implicitly eliminates generation of twins (similar conformations) in the population; 4) the identification of a meme (protein substructures) during optimization from different basins of attraction - a process that is equivalent to implicit applications of threading principles; 5) a clustering of the population corresponds to basins of attraction that allows evolution to overcome the complexity of multimodal search space, thereby avoiding search getting trapped in a local optimum; and 6) a 2-stage framework gathers domain knowledge (i.e., substructures or memes) from different basins of attraction for a combined execution in the second stage. Experiments conducted with different lattice models using known benchmark protein sequences and comparisons carried out with recently reported approaches in this journal show that the proposed algorithm has robustness, speed, accuracy, and superior performance. The approach is generic and can easily be extended for applications to other classes of problems.

Multiclass microarray gene expression classification based on fusion of correlation features

- Chetty, Girija, Chetty, Madhu

  • Authors: Chetty, Girija , Chetty, Madhu
  • Date: 2010
  • Type: Text , Conference paper
  • Relation: Information Fusion (FUSION), 2010 13th Conference
  • Full Text: false
  • Reviewed:
  • Description: In this paper, we propose novel algorithmic models based on fusion of independent and correlated gene features for multiclass microarray gene expression classification. It is possible for genes to get co-expressed via different pathways. Moreover, a gene may or may not be co-active for all samples. In this paper, we approach this problem with a optimal feature selection technique using analysis based on statistical techniques to model the complex interactions between genes. The two different types of correlation modelling techniques based on the cross modal factor analysis (CFA) and canonical correlation analysis (CCA) were examined. The subsequent fusion of CCA/CFA features with principal component analysis (PCA) features at feature-level, and at score-level result in significant enhancement in classification accuracy for different data sets corresponding to multiclass microarray gene expression data.

Degree of differential prioritization

- Ooi, Chia, Chetty, Madhu, Teng, Shyh

  • Authors: Ooi, Chia , Chetty, Madhu , Teng, Shyh
  • Date: 2009
  • Type: Text , Journal article
  • Relation: IEEE Engineering in Medicine and Biology Magazine Vol. 28, no. 4 (2009), p. 45-51
  • Full Text: false
  • Reviewed:
  • Description: Because of the high dimensionality of the microarray data sets, feature selection (FS) has become an important challenge in molecular classification. Using the degree of differential prioritization (DDP) between relevance and antiredundancy, our proposed DDP-based FS technique is capable of achieving better accuracies than those previously reported, using a smaller predictor set. However, previously, we have neither devised nor used any method for determining the value of the DDP to be used for the data set of interest before the FS process. In this article, we propose a system for predicting the optimal value of the DDP, which costs less computationally than conventional tuning while maintaining the independence of the FS technique from the type of underlying classifier used

DFS based partial pathways in GA for protein structure prediction

- Hoque, Md Tamjidul, Chetty, Madhu, Lewis, Andrew, Sattar, Abdul

  • Authors: Hoque, Md Tamjidul , Chetty, Madhu , Lewis, Andrew , Sattar, Abdul
  • Date: 2008
  • Type: Text , Conference paper
  • Relation: Third IAPR International Conference, PRIB 2008
  • Full Text:
  • Reviewed:
  • Description: Nondeterministic conformational search techniques, such as Genetic Algorithms (GAs) are promising for solving protein structure prediction (PSP) problem. The crossover operator of a GA can underpin the formation of potential conformations by exchanging and sharing potential sub-conformations, which is promising for solving PSP. However, the usual nature of an optimum PSP conformation being compact can produce many invalid conformations (by having non-self-avoiding-walk) using crossover. While a crossover-based converging conformation suffers from limited pathways, combining it with depth-first search (DFS) can partially reveal potential pathways. DFS generates random conformations increasingly quickly with increasing length of the protein sequences compared to random-move-only-based conformation generation. Random conformations are frequently applied for maintaining diversity as well as for initialization in many GA variations.

Novel local improvement techniques in clustered memetic algorithm for protein structure prediction

- Islam, Md Kamrul, Chetty, Madhu, Murshed, Manzur

  • Authors: Islam, Md Kamrul , Chetty, Madhu , Murshed, Manzur
  • Date: 2011
  • Type: Text , Conference paper
  • Relation: IEEE Congress on Evolutionary Computation (IEEE CEC) p. 1003-1011
  • Full Text: false
  • Reviewed:
  • Description: Evolutionary algorithms (EAs) often fail to find the global optimum due to genetic drift. As the protein structure prediction problem is multimodal having several global optima, EAs empowered with combined application of local and global search e.g., memetic algorithms, can be more effective. This paper introduces two novel local improvement techniques for the clustered memetic algorithm to incorporate both problem specific and search-space specific knowledge to find one of the optimum structures of a hydrophobic-polar protein sequence on lattice models. Experimental results show the superiority of the proposed techniques against existing EAs on benchmark sequences.

Constraint minimization for efficient modelling of gene regulatory network

- Ram, Ramesh, Chetty, Madhu, Bulach, Dieter

  • Authors: Ram, Ramesh , Chetty, Madhu , Bulach, Dieter
  • Date: 2008
  • Type: Text , Conference paper
  • Relation: Third IAPR International Conference, PRIB 2008 Melbourne
  • Full Text: false
  • Reviewed:
  • Description: Due to various complexities, as well as noise and high dimensionality, reconstructing a gene regulatory network (GRN) from a high-throughput microarray data becomes computationally intensive.In our earlier work on causal model approach for GRN reconstruction, we had shown the superiority of Markov blanket (MB) algorithm compared to the algorithm using the existing Y and V causal models. In this paper, we show the MB algorithm can be enhanced further by application of the proposed constraint logic minimization (CLM) technique. We describe a framework for minimizing the constraint logic involved (condition independent tests) by exploiting the Markov blanket learning methods developed for a Bayesian network (BN). The constraint relationships are represented in the form of logic using K-map and with the aid of CLM increase the algorithm efficiency and the accuracy. We show improved results by investigations on both the synthetic as well as the real life yeast cell cycle data sets.

A study on the importance of differential prioritization in feature selection using toy datasets

- Ooi, Chia, Teng, Shyh, Chetty, Madhu

  • Authors: Ooi, Chia , Teng, Shyh , Chetty, Madhu
  • Date: 2008
  • Type: Text , Conference paper
  • Relation: Third IAPR International Conference, PRIB
  • Full Text: false
  • Reviewed:
  • Description: Previous empirical works have shown the effectiveness of differential prioritization in feature selection prior to molecular classification. We now propose to determine the theoretical basis for the concept of differential prioritization through mathematical analyses of the characteristics of predictor sets found using different values of the DDP (degree of differential prioritization) from realistic toy datasets. Mathematical analyses based on analytical measures such as distance between classes are implemented on these predictor sets. We demonstrate that the optimal value of the DDP is capable of forming a predictor set which consists of classes of features which are well separated and are highly correlated to the target classes – a characteristic of a truly optimal predictor set. From these analyses, the necessity of adjusting the DDP based on the dataset of interest is confirmed in a mathematical manner, indicating that the DDP-based feature selection technique is superior to both simplistic rank-based selection and state-of-the-art equal-priorities scoring methods. Applying similar analyses to real-life multiclass microarray datasets, we obtain further proof of the theoretical significance of the DDP for practical applications