Generally, in neuralnetworks thesparsenessisasuitableregularizer in a lot of applications. In this paper, sparse connectivity and sparse representation are used to enhance solutions to the problem of classification. Sequential feature selection is then leveraged to remove redundant features and select relevant ones. Sparseness-enforcing projection operator is used to discovering the most similar vector with a predefined sparseness degree for any input vector as well. As it will be argued, the mentioned operator is approximately differentiable at every point. From the facts it is clear that the sparseness enforcing projection would be appropriate for use as a transfer function in the proposed neural network and the network can be tuned using gradient based methods. Meanwhile, an intelligent method was usedtobuild the architecture of the proposed neuralnetworkto achievebetterperformance.TheMNISTdatasetwhichconsistsof 70,000 handwritten digits was used to train and test the method and 99.18% accuracy was achieved by classifying this dataset.
