In this paper we propose a novel technique of using a hybrid evolutionary method, which uses a combination of genetic algorithm and matrix based solution methods such as QR factorization. The training of the model is based on a layer based hierarchical structure for the architecture and the weights for the Artificial Neural Network classifier. The architecture for the classifier is found using a binary search type procedure. The hierarchical structured algorithm (EALS-BT) is also a hybrid, because it combines the Genetic Algorithm based method with the Matrix based solution method for finding weights. A heuristic segmentation algorithm is initially used to over segment each word. Then the segmentation points are passed through the rule-based module to discard the incorrect segmentation points and include any missing segmentation points. Following the segmentation the contour is extracted between two correct segmentation points. The contour is passed through the feature extraction module that extracts the angular features, after which the EALS-BT algorithm finds the architecture and the weights for the classifier network. These recognized characters are grouped into words and passed to a variable length lexicon that retrieves words that have the highest confidence value.