An electrocardiogram (ECG) is an essential piece of medical equipment that helps diagnose various heart-related conditions in patients. An automated diagnostic tool is required to detect significant episodes in long-term ECG records. It is a very challenging task for cardiologists to analyze long-term ECG records in a short time. Therefore, a computer-based diagnosis tool is required to identify crucial episodes. Myocardial infarction (MI) and conduction disorders (CDs), sometimes known as heart blocks, are medical diseases that occur when a coronary artery becomes fully or suddenly stopped or when blood flow in these arteries slows dramatically. As a result, several researchers have utilized deep learning methods for MI and CD detection. However, there are one or more of the following challenges when using deep learning algorithms: (i) struggles with real-life data, (ii) the time after the training phase also requires high processing power, (iii) they are very computationally expensive, requiring large amounts of memory and computational resources, and it is not easy to transfer them to other problems, (iv) they are hard to describe and are not completely understood (black box), and (v) most of the literature is based on the MIT-BIH or PTB databases, which do not cover most of the crucial arrhythmias. This paper proposes a new deep learning approach based on machine learning for detecting MI and CDs using large PTB-XL ECG data. First, all challenging issues of these heart signals have been considered, as the signal data are from different datasets and the data are filtered. After that, the MI and CD signals are fed to the deep learning model to extract the deep features. In addition, a new custom activation function is proposed, which has fast convergence to the regular activation functions. Later, these features are fed to an external classifier, such as a support vector machine (SVM), for detection. The efficiency of the proposed method is demonstrated by the experimental findings, which show that it improves satisfactorily with an overall accuracy of 99.20% when using a CNN for extracting the features with an SVM classifier.