Early and accurate detection of dysplasia in colorectal polyps can improve prognosis and increase survival chances. Recently, automated learning-based approaches using histopathological images have been adopted for improved classification of polyps. The supervised learning approaches do not provide a reliable classification performance due to limited annotated samples. But, in unsupervised learning, some hidden features are extracted from the unlabeled data which may not be effective in discriminating the complex patterns of the dataset. A generative adversarial network (GAN) is proposed in this work based on a semi-supervised framework for colorectal polyp classification using histopathological images. Our framework learns the discriminating features in an adversarial manner from the limited labeled and huge unlabeled data. In the supervised mode, the discriminator of the proposed model is trained to classify the real histopathological images, whereas, in the unsupervised mode, it tries to discriminate between real and fake images, similar to the classical GAN network. By training in unsupervised mode, the discriminator can identify and extract the subtle features from unlabeled images, to develop a generalized robust model. Our technique yielded classification accuracies of 87.50% and 76.25% using 25% and 50% majority voting schemes, respectively, on the UniToPatho dataset.