Information extraction forms a very large and important component of NLP research which aims at extracting varying nature of information from a text corpus. This information could vary from (named) entities and their inter-relationships in sentences to facts which could later be used for different tasks like search engine retrieval, question answering, etc. Most of these tasks and their associated (primarily) machine learning based solutions ultimately hit a roadblock due to the lack of manually labeled data complemented by an expensive and laborious annotation task. While unsupervised/semi-supervised methods can be developed for these tasks, their effectiveness and usability could be compromised. For the task of relation extraction, the distant supervised paradigm has been shown to have enormous potential in providing a relatively very large amount of training data, at the cost of label noise. Prior efforts have proposed a variety of solutions to reduce the impact of label noise both at an architectural level, as well as by adding a small amount of manual supervision. However, we aim to explore a different relation extraction paradigm - can distant supervision help to improve supervised neural relation extraction? This thesis focuses on exploring various strategies such that a supervised relation extraction model, when supplemented with distant supervision is able to perform better at test time. While we are unable to successfully use approaches based on an attention driven subspace alignment and adversarial training for our goal, a simple distillation based approach can result in an improvement in the model’s performance.