Electrocardiography (ECG) is an essential tool for understanding the electrophysiology of the heart and its electrical activity in response to cardiovascular diseases (CVD). However, a physical paper-based format is common, which poses difficulties in the presence of millions of patients’ data, reducing global accessibility in cardiac assessment. We propose a simple approach to digitize physical ECGs (images or papers) and a novel deep learning to classify the CVD condition (normal or abnormal) using a convolutional neural network (CNN). Digitization simply involves removing the background and reading pixels on an image mask to identify a signal. On the other hand, the CNN includes feed-forward blocks to train on automatically extracted image features. Our team, VitalRhythms, achieved a challenge classification score (F-measure) of 0.694 during the unofficial phase and an overall 10-fold cross-validation score of 0.766. Moreover, the reconstruction score (SNR) was -18.10, with a 10-fold cross-validation SNR of -13.20, but our official phase training submissions were unsuccessful. This study paves the way towards implementing sophisticated deep learning tools for the purpose of digitizing paper-based ECG and aiding the assessment of cardiovascular diseases and, thus, simplifying cardiac care in the presence of big patient data.