Information Theory (Fall 2020)



Class meetings: Tuesday 9:15 -12:00 (first time on September 15, 2020)





The material presented at each lecture is summarized in the lecture notes below that are updated after each lecture. Parts of that text is also used during the online lectures. (If some other outside source is used during the lecture I will also link it below these notes if possible.)

Lecture notes (Fall 2020)

On the first lecture we also watched a short trailer of The Bit Player which is a film about Claude Shannon. We looked at his famous diagram, too, that can be seen on this page.

The following are supplementary files used on the seventh lecture (on October 27), when we learnt about universal source coding.

Examples for the Lempel-Ziv data compression algorithms LZ77 and LZ78 (Source: Györfi László, Gyôri Sándor, Vajda István "Információ- és kódelmélet", textbook in Hungarian, published by Typotex Kiadó)

Example for the Lempel-Ziv-Welch data compression algorithm (Source: Györfi László, Gyôri Sándor, Vajda István "Információ- és kódelmélet", textbook in Hungarian, published by Typotex Kiadó)



Midterms: The first midterm will be held online on the Moodle platform on October 21, 2020, starting at 11:00 a.m. (Budapest time). There will also be a second midterm in the semester and the final grade will be calculated from the results of these two. To obtain a valid grade it is required, however, that you achieve a passing grade (that is, at least a 2) on both midterms separately. Details (including the timing) of the second midterm will be announced later. For both midterms there will be a retake option if you wish to improve on your grade obtained first time. (The timing of these will also be announced later.) If you write the retake of a midterm, however, then the result of the retake will be taken into account when calculating your final grade. One important exception is, that in case you already have a passing grade on a midterm, then even if you take the retake and fail on it, it will be considered so that you already passed at that midterm. But in that case, its result is taken into account as if you have just passed.

Below are the details of the technicalities for the first midterm.

The first midterm will have two parts. The first part is a quiz of ten questions with multiple choice answers (four options for each) that you have 25 minutes to fill. (Please note that once you answered a question and moved on for the next one you will not be able to step back and modify a previous answer.) After this quiz is over, the system will calculate your score and depending on that you can have the following choices. In case you answered at least 6 questions correctly, you have the right to continue to the second part but you can also stop and accept the grade that is offered at this stage. The offered grade is a 3 if you answered at least 8 questions correctly and it is a 2 if you answered at least 6 but less than 8 questions correctly. (In case the number of your correct answers is fewer than 6 then unfortunately you failed and should write the retake midterm at a later date for obtaining a valid grade.) That is at this point if you have at least 6 correct answers you can finish the exam and get the offered grade but that cannot be better than a 3. You will have 15 minutes to decide whether you continue or not. You have to know, however, that once you register for the second part, your result becomes open again: you can both worsen and improve the previously offered grade. However, you can worsen by at most 1 grade and improve by at most 2 grades.

In case you want to continue, you will have to register for it at the place the system will offer for you. The second part is an assignment consisting of three exercises that you have to solve and submit the written solutions as a pdf file. You will have 60 minutes for the second part.

Both parts are open notes, that is, you are allowed to use your notes. (You are asked, however, not to communicate to anyone during the midterm, that is not considered acceptable.)

The scores you can get in the second part are as follows. Each of the three exercises it contains worth 5 points if completely solved, so altogether you can get 15 points for them. If you get below 4 points then your grade is worsened by 1. If you get at least 4 but less than 8 points then your grade remains what was offered to you after the quiz. If you achieve at least 8 points but less then 12 points than your grade improves by 1. Finally, if you achieve at least 12 points then your grade improves by 2.

A technical remark concerning the second part: If it is possible for you then please send scanned files instead of photos. For this please note that there are applications downloadable to smartphones that can turn a smartphone into a scanner. (E..g. CamScan is such an application, but there are certainly other ones, too, I mention this one only because that is what I heard about the most. In case you cannot do this, I will accept the photos turned to pdf but generally they are of much worse quality, that is why I ask you to avoid that if possible.

SAMPLE SHORTER QUIZ: A sample shorter quiz (5 questions only) is made available on Moodle to give you an opportunity to test the system and get an impression about the type of quiz questions before the midterm. This can be filled once any time in one shot within 14 minutes. I recommend you do it also for getting acquainted with the system.

RETAKE of the FIRST MIDTERM: We fixed the time of this retake to be on November 4, 2020, starting at 11:00 a.m. (according to Budapest time). The sheme and rules of this retake are the same as for its original version on October 21. (Naturally, the material of the retake is also the same as that of the original, that is, it will not include what was covered only on lectures since then.)

SECOND MIDTERM: The second midterm will be held on December 2, Wednesday starting at 11:00 a.m. (Budapest time). It will also be online on the Moodle platform and its form will be similar to that of the first midterm (two parts, first a quiz and then an assignment for those who pass the quiz and choose to continue, see detailed description above for the first midterm) as well as the rules. (There will be one small difference: the time for the quiz will now be 30 minutes as opposed to only 25 minutes in case of the first midterm.)

RETAKE of SECOND MIDTERM: The retake of the second midterm will be held exactly one week after the second midterm, that is, on December 9 starting at 11:00 a.m. (Budapest time) in similar form and with similar rules as the second midterm, see details above.

GRADING: As already written above you have to pass both midterms separately to obtain a valid grade. Once this is successfully done, your grade is calculated as follows. If the average of the two (passing) grades you obtained on the two midterms is an integer, then that will be your course grade. If it is not an integer (in which case it will be an integer +0.5 as it is the average of two integers), then it is checked for what score you received the grades on the midterms. In case at least one of the grades is obtained with a strictly better score than the absolute minimum required for that grade, then the mentioned average is rounded upwards when calculating your course grade. If both midterm grades are obtained by the minimum possible score for that grade then their average is rounded downwards when your course grade is calculated.

FINAL MAKE UP OPTION after the end of teaching period for those who passed one midterm but not the other: In case you failed one of the two midterms but passed the other one, there will be a final opportunity after the end of the teaching period to have a second retake option of that midterm on which you did not pass. Unfortunately this option is not open for those who failed on both midterms, in that case a valid course grade already cannot be obtained. According to university regulations this final option can also be used for trying to improve the result of one of the midterms for those who passed both midterms. However, in that case it will be the result achieved at this opportunity that will count as the valid result of that midterm even if it is a fail. That is, here the already passing grade is not guaranteed to be preserved as it was for the first retake option. So this is important: If someone registers for this final retake (even if only for improvement) and fails on it, then that person fails the course and cannot get a valid grade. Note also that participation on this final option requires registration and even the payment of a fee for the education office. (There might be some exceptional cases for the fee, ask the education office if you believe you might be eligible for that.) The time of this final option will be December 16, 11:00 a.m. for both midterms. (Be careful to write the one you need to retake, otherwise it cannot substitute the result of that one.) The scheme is the same for both midterms as at their original writing.