• MVM

    The Mini VM.

  • KA Lite Dev

    Some stuff in KA Lite that I devved.

  • The beginnings

  • Picking up where i left off - I’m now writing the mark and sweep algorithm.

  • Starting with the “mark” phase - I need to add another attribute into each object to mark it as “reachable”.

  • Hehey! This should be easy. Changes are in this commit: https://github.com/aronasorman/mlcsv/commit/8359ca3e80451b49f9215006a68bfcb632d75040

  • But hey! It’s not working. It’s not writing out the exam jsons. Rather than doing the normal insert-debugger-line-here method I normally do in python, I’m gonna try using strace to see what’s wrong.

  • I straced the executable and it seems that it doesn’t write the expected files at all. Which was totally what I expected :P looks I’m gonna have to use the normal debugging techniques again.

  • Turns out the problem is two-fold: the Exam and playlist IDs in the new CSVs weren’t in their right places, and we weren’t processing lines with “Mid Year Exam” as an exam. Fixed those in 3c87d1c

  • OK! Goal for now is to jump off Richard’s audio content html page, remove his stuff, add in PDF.js, then make it work.

  • I’ve downloaded the “build” version of PDF.js which includes a viewer.html and some static assets. I’m looking into integrating that wholesale into KA Lite, then just ask Dylan/Jessica to integrate it.

  • First step: just added pdf.js and pdf.worker.js into the static-libraries.

  • I don’t have any code yet, but i’m stuck at trying to make the topic tree load a PDF. Now how would I do that? I’m looking at the JS stack trace to see how the whole thing loads.

  • I just faked it by making a PDF viewer load when kind == “Video”. Heh. Then just load a static pdf rather than what the actual node is. On to making the PDF viewer work!

  • I’m considering just embedding the builtin pdf.js viewer.html inside an iframe for the prototype. Let’s see how this goes.

  • So simply embedding it in an iframe seems to be working well, except for the fact that it’s too small right now. Next step is to find a way to resize it dynamically based on the window size. Bootstrap can hopefully help in that aspect.

  • And oh reminder to myself: stick PDFJS.disableFontFace = true into the PDF.js initialization code so that it will work on mobile browsers.

  • OK, tried it out with the test tablet from China. Loading is a bit slow with the PDF (about 5-7 seconds) but after that there seems to be no bugs in rendering. Yahoo!

  • Ok, moving to PDF progress tracking now. I’m skimming through the viewer.js and other files bundled by Mozilla (which I stuck inside the iframe described above). And I saw this: https://github.com/mozilla/pdf.js/blob/master/web/viewer.js#L277

    Looks helpful for tracking the user’s progress on a PDF.

  • I tried accessing PDFVIew at which watchScroll is defined, but it’s not defined as it’s named. It’s probably defined somewhere as a variable. Will continue reading.

  • For now it seems to much work to log every page they visited, so we’re going with checking if they opened the PDF for X seconds.

  • So to do any kind of logging at all I’m gonna have to find a way to access the javascript variables inside the iFrame. Researching that now.

  • So to access the window object of an iframe, you use this.contentWindow. Nifty.

  • There’s also the DOMContentLoaded event. Will investigate how this is used.

  • So I made the skeleton frontend code that listens to the pagechange event and the pageshow event to track progress. Doesn’t do any actual API calls yet: https://github.com/aronasorman/ka-lite/commit/0ab9f7b6548dbd312ce8a440258bf3542240bae3

    I’m gonna work on tracking progress right now through the ContentLog.

  • So to fetch a ContentLog we have to ensure that the user is logged in. I just copy pasted richard’s code to mine: https://github.com/aronasorman/ka-lite/blob/pdfjs/kalite/distributed/static/js/distributed/audio/views.js#L15-L30

    Now, this common code can be shared. I prefer a mixin, as recommended here: http://ricostacruz.com/backbone-patterns/#mixins

    I’ll hold off on doing that though. I’ll confer with Richard and Jamie on what they prefer. For now I just copy pasted that snippet.

  • So some of the time an event’s target is the iframe’s document itself. One of the difficulties I was having was how to access the window object from the document. To do that, I just had to call document.defaultView and that would return the appropriate window object.

  • Woohoo content logging is done! Now on to setting points.

  • Point tracking done: https://github.com/aronasorman/ka-lite/commit/1f647ffd8b7b37c6caf8345d67019a1141490926

    Some gotchas:

  • The task is to have a “clear test” UI for teachers so they can retake it.

  • Backend wise, seems like the easiest way is to just delete the test log, then make an endpoint for it, and finally a button.

  • Lots of refactoring work for the StudentTestTest. Need to update it to use mixins.

  • As well as update it to use the new Test.total_questions attribute.

  • Now apparently some old call of self.create_facility is failing. Due to changed forms again?

  • Specifically, this line: https://github.com/aronasorman/ka-lite/blob/nalanda-rct3/kalite/distributed/tests/browser_tests/base.py

    With this error:

    Traceback (most recent call last):
      File "/home/aron/src/ka-lite/kalite/distributed/tests/browser_tests/student_testing.py", line 44, in setUp
        super(StudentTestTest, self).setUp()
      File "/home/aron/src/ka-lite/kalite/distributed/tests/browser_tests/base.py", line 207, in setUp
        self.facility = self.create_facility(facility_name=self.facility_name)
      File "/home/aron/src/ka-lite/kalite/testing/mixins/facility_mixins.py", line 17, in create_facility
        obj, created = Facility.objects.get_or_create(**fields)
      File "/home/aron/src/ka-lite/python-packages/django/db/models/manager.py", line 146, in get_or_create
        return self.get_query_set().get_or_create(**kwargs)
      File "/home/aron/src/ka-lite/python-packages/django/db/models/query.py", line 470, in get_or_create
        return self.get(**lookup), False
      File "/home/aron/src/ka-lite/python-packages/django/db/models/query.py", line 379, in get
        clone = self.filter(*args, **kwargs)
      File "/home/aron/src/ka-lite/python-packages/django/db/models/query.py", line 655, in filter
        return self._filter_or_exclude(False, *args, **kwargs)
      File "/home/aron/src/ka-lite/python-packages/django/db/models/query.py", line 673, in _filter_or_exclude
        clone.query.add_q(Q(*args, **kwargs))
      File "/home/aron/src/ka-lite/python-packages/django/db/models/sql/query.py", line 1266, in add_q
        can_reuse=used_aliases, force_having=force_having)
      File "/home/aron/src/ka-lite/python-packages/django/db/models/sql/query.py", line 1134, in add_filter
        process_extras=process_extras)
      File "/home/aron/src/ka-lite/python-packages/django/db/models/sql/query.py", line 1332, in setup_joins
        "Choices are: %s" % (name, ", ".join(names)))
    FieldError: Cannot resolve keyword 'facility_name' into field. Choices are: address, address_normalized, contact_email, contact_name, contact_phone, counter, deleted, description, facilitygroup, facilityuser, id, latitude, longitude, name, signature, signed_by, signed_version, user_count, zone_fallback, zoom
  • Just removed that line. Heh.

  • Ok, I seem to be getting an unrelated Selenium error, with Firefox crashing with this message:

    Traceback (most recent call last):
      File "/home/aron/src/ka-lite/kalite/testing/browser.py", line 174, in setUp
        (self.browser, self.admin_user, self.admin_pass) = setup_test_env(browser_type=browser_type)
      File "/home/aron/src/ka-lite/kalite/testing/browser.py", line 38, in setup_test_env
        local_browser = getattr(webdriver, browser_type)()  # Get local session of browser
      File "/home/aron/src/ka-lite/python-packages/selenium/webdriver/firefox/webdriver.py", line 59, in __init__
        self.binary, timeout),
      File "/home/aron/src/ka-lite/python-packages/selenium/webdriver/firefox/extension_connection.py", line 47, in __init__
        self.binary.launch_browser(self.profile)
      File "/home/aron/src/ka-lite/python-packages/selenium/webdriver/firefox/firefox_binary.py", line 61, in launch_browser
        self._wait_until_connectable()
      File "/home/aron/src/ka-lite/python-packages/selenium/webdriver/firefox/firefox_binary.py", line 105, in _wait_until_connectable
        self.profile.path, self._get_firefox_output()))
    WebDriverException: Message: 'Can\'t load the profile. Profile Dir: /tmp/tmpbXa7eJ Firefox output: \n(process:27038): GLib-CRITICAL **: g_slice_set_config: assertion \'sys_page_size == 0\' failed\n1410068812152\taddons.manager\tDEBUG\tLoaded provider scope for resource://gre/modules/addons/XPIProvider.jsm: ["XPIProvider"]\n1410068812154\taddons.manager\tDEBUG\tLoaded provider scope for resource://gre/modules/LightweightThemeManager.jsm: ["LightweightThemeManager"]\n1410068812156\taddons.xpi\tDEBUG\tstartup\n1410068812157\taddons.xpi\tINFO\tMapping online-accounts@lists.launchpad.net to /usr/lib/mozilla/extensions/{ec8030f7-c20a-464f-9b0e-13a3a9e97384}/online-accounts@lists.launchpad.net\n1410068812158\taddons.xpi\tINFO\tMapping {2e1445b0-2682-11e1-bfc2-0800200c9a66} to /usr/share/mozilla/extensions/{ec8030f7-c20a-464f-9b0e-13a3a9e97384}/{2e1445b0-2682-11e1-bfc2-0800200c9a66}\n1410068812158\taddons.xpi\tINFO\tMapping webapps-team@lists.launchpad.net to /usr/share/mozilla/extensions/{ec8030f7-c20a-464f-9b0e-13a3a9e97384}/webapps-team@lists.launchpad.net\n1410068812158\taddons.xpi\tINFO\tMapping ubufox@ubuntu.com to /usr/share/mozilla/extensions/{ec8030f7-c20a-464f-9b0e-13a3a9e97384}/ubufox@ubuntu.com\n1410068812158\taddons.xpi\tINFO\tMapping {972ce4c6-7e08-4474-a285-3208198ce6fd} to /usr/lib/firefox/browser/extensions/{972ce4c6-7e08-4474-a285-3208198ce6fd}\n1410068812158\taddons.xpi\tINFO\tMapping langpack-en-ZA@firefox.mozilla.org to /usr/lib/firefox/browser/extensions/langpack-en-ZA@firefox.mozilla.org.xpi\n1410068812159\taddons.xpi\tINFO\tMapping langpack-en-GB@firefox.mozilla.org to /usr/lib/firefox/browser/extensions/langpack-en-GB@firefox.mozilla.org.xpi\n1410068812159\taddons.xpi\tINFO\tMapping fxdriver@googlecode.com to /tmp/tmpbXa7eJ/extensions/fxdriver@googlecode.com\n1410068812160\taddons.xpi\tDEBUG\tcheckForChanges\n1410068812171\taddons.xpi\tDEBUG\tNo changes found\n1410068812176\taddons.xpi\tDEBUG\tRegistering manifest for /usr/lib/firefox/browser/extensions/langpack-en-ZA@firefox.mozilla.org.xpi\n1410068812177\taddons.xpi\tDEBUG\tRegistering manifest for /usr/lib/firefox/browser/extensions/langpack-en-GB@firefox.mozilla.org.xpi\n*** Blocklist::_preloadBlocklistFile: blocklist is disabled\n'

    Ugh. What will I do to make this test run?

  • Debugging that firefox error later. Dropped back to phantomjs. Getting this error now:

    ERROR: test_exercise_mastery (kalite.distributed.tests.browser_tests.student_testing.StudentTestTest)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/home/aron/src/ka-lite/kalite/distributed/tests/browser_tests/student_testing.py", line 46, in setUp
        self.fac = self.create_facility(self.facility_name)
    TypeError: create_facility() takes exactly 1 argument (2 given)

    Seems like we’re still getting the old “create_facility” defined in FacilityTestCase. How to skip that?

  • Turns out create_facility only accepts KEYWORD arguments, and we’re passing the facility as a POSITIONAL argument. The correct line should be:

     self.fac = self.create_facility(name=self.facility_name)
  • Now we’re getting a timeout:

    ======================================================================
    ERROR: test_exercise_mastery (kalite.distributed.tests.browser_tests.student_testing.StudentTestTest)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/home/aron/src/ka-lite/kalite/distributed/tests/browser_tests/student_testing.py", line 85, in test_exercise_mastery
        self.browser_submit_answers(answer)
      File "/home/aron/src/ka-lite/kalite/distributed/tests/browser_tests/student_testing.py", line 67, in browser_submit_answers
        (By.CSS_SELECTOR, "#solutionarea > input[type=text]")))
      File "/home/aron/src/ka-lite/python-packages/selenium/webdriver/support/wait.py", line 71, in until
        raise TimeoutException(message)
    TimeoutException: Message: ''
  • It’s failing with some UI exception, but I can’t see the error because I’m running phantomjs. Time to make Firefox work.

  • as per this stack overflow answer: http://stackoverflow.com/a/25645344

    I had to downgrade my version of Firefox to version 30 to make everything work again.

  • Ran the test_exercise_mastery test again. Success.

  • Ran all tests. Failure, with half of them coming from the deletion of the default facility for KALiteDistributedBrowserTestCase. Fixing.

  • Ok, first off develop doesn’t have a search bar anymore. So I added it back with this code:

    {# Search box #}
                                <li>
                                  <form class="navbar-form" action="{% url 'search' %}" method="get" id="search-box" role="search">
                                    {% comment %}translators: this will appear in the navigation bar. please try to keep it as short as possible.{% endcomment %}
                                    <div class="input-group">
                                      <input type="text" name="query" id="search" class="form-control" placeholder="{% trans 'video, topic, or exercise...' %}" />
                                      <button class="btn btn-default" type="submit"><i class="glyphicon glyphicon-search"></i></button>
                                    </div>
                                  </form>
                                </li>
  • Second, I’m gonna have to make the JS code in search_autocomplete.js at least find that search box so I can see what endpoint is being called.

  • NVM. Turns out it’s already detecting it once I put a breakpoint on fetchTopicTree. Now I gotta extend that API endpoint to return other content types as well.

  • Querying the api directly on the browser (with http://127.0.0.1:8008/api/flat_topic_tree/en) resulted in this JSON:

    {
    Topic: {
    demo: {
    available: true,
    path: "demo/",
    kind: "Topic",
    title: "demo"
    },
    classrooms: {
    available: true,
    path: "demo/classrooms/",
    kind: "Topic",
    title: "classrooms"
    },
    physics: {
    available: true,
    path: "demo/science/physics/",
    kind: "Topic",
    title: "physics"
    },
    math: {
    available: true,
    path: "demo/math/",
    kind: "Topic",
    title: "math"
    },
    science: {
    available: true,
    path: "demo/science/",
    kind: "Topic",
    title: "science"
    }
    },
    Video: {
    3db321ccaec3154139dd42ded78f19b5: {
    available: true,
    path: "demo/classrooms/healing-classrooms-1",
    kind: "Video",
    title: "Healing Classrooms 1"
    }
    },
    Exercise: { }
    }

    So it’s mostly working already, I just need to extend it to add the other content types. Whoo! Thought I need to do more hackzor.

  • Alright so I actually had to edit generate_node_cache to add in other types, like so: https://github.com/aronasorman/ka-lite/blob/search/kalite/topic_tools/__init__.py#L259-L260

    I also had to edit get_content_cache to be able to filter the content: https://github.com/aronasorman/ka-lite/blob/search/kalite/topic_tools/__init__.py#L113-L126

    So now the API endpoint returns the content kinds we explicitly support! Nice!

  • So I really didn’t do much, and the search autocomplete is now working already. Problems I saw:

    • No icons for the new content types
    • The separate search page isn’t styled correctly
    • Pressing enter for text like “math” leads to a 404.
  • Passed off to Richard.

  • So first step is to make a skeleton management command. I used NoArgsCommand since I don’t expect any argument to this.

  • Proceeding to make a test for this. The basic aim of the test is to see if we’re making a request to the right url. So it will involve the mock library.

  • Mocking requests might be easier with httreplay. Looking into it to see if it’s easy to work with.

  • I saw this cool library called vcr.py. Kinda like vcr from the Rails community. I’m thinking of switching to this from manually constructing request mock objects as it will be easier to keep up to date with the remote server.

  • I’m thinking of making a wrapper for vcr.py so that testers don’t have to fudge with the cassette names. They just call a with statement (or a decorator) and boom!

  • The wrapper will be the new KALiteTestCase. Lelz.

  • I had KALiteTestCase inherit from CreateDeviceMixin and TestCase. That caused a couple of MRO resolution errors, which I promptly fixed. From KALiteTestCase I made a KALiteClientTestCase which is for Django client tests, and a KALiteBrowserTestCase which is for browser tests.

    I’m encountering some test failures now with student_testing.CoreTests, which I modified to inherit from KALiteClientTestCase. The errors look something like:

    Traceback (most recent call last):
      File "/home/aron/src/ka-lite/kalite/student_testing/tests.py", line 81, in setUp
        super(CoreTests, self).setUp()
      File "/home/aron/src/ka-lite/kalite/student_testing/tests.py", line 38, in setUp
        self.assertTrue(self.client.facility)
    AssertionError: None is not True
  • Fixed. Just had CoreTests and CurrentUnitTests inherit from KALiteClientTestCase.

  • Now fixing more test breakage caused by MRO errors (a test case inheriting from both CreateDeviceMixin and KALiteTestCase, which itself inherits CreateDeviceMixin).

  • Now getting this error when running all the tests:

    Traceback (most recent call last):
      File "/home/aron/src/ka-lite/python-packages/django/core/management/base.py", line 222, in run_from_argv
        self.execute(*args, **options.__dict__)
      File "/home/aron/src/ka-lite/python-packages/django/core/management/commands/test.py", line 80, in execute
        super(Command, self).execute(*args, **options)
      File "/home/aron/src/ka-lite/python-packages/django/core/management/base.py", line 255, in execute
        output = self.handle(*args, **options)
      File "/home/aron/src/ka-lite/python-packages/south/management/commands/test.py", line 8, in handle
        super(Command, self).handle(*args, **kwargs)
      File "/home/aron/src/ka-lite/python-packages/django/core/management/commands/test.py", line 106, in handle
        failures = test_runner.run_tests(test_labels)
      File "/home/aron/src/ka-lite/kalite/testing/testrunner.py", line 68, in run_tests
        return run_tests_wrapper_fn()
      File "/home/aron/src/ka-lite/python-packages/django/test/utils.py", line 220, in inner
        return test_func(*args, **kwargs)
      File "/home/aron/src/ka-lite/kalite/testing/testrunner.py", line 67, in run_tests_wrapper_fn
        return super(KALiteTestRunner,self).run_tests(test_labels, extra_tests, **kwargs)
      File "/home/aron/src/ka-lite/python-packages/django/test/simple.py", line 366, in run_tests
        suite = self.build_suite(test_labels, extra_tests)
      File "/home/aron/src/ka-lite/kalite/testing/testrunner.py", line 79, in build_suite
        testfun = getattr(test, test._testMethodName)
    AttributeError: 'TestSuite' object has no attribute '_testMethodName'

    Investigating.

  • Only happens on the code called when --failfast is on. Something related to these lines, found in:

            if self.failfast:
                for test in test_suite._tests:
                    testfun = getattr(test, test._testMethodName)
                    setattr(test, test._testMethodName, auto_pdb()(testfun))
            return test_suite
  • ….
    I got into this refactoring blackhole. Coming back out with refactored tests!

  • While rewriting the tests, I decided it would be nice to have a multiple cursor feature like sublime text. So I just downloaded the multiple-cursors library and integrated it into Emacs.

  • Back! Turns out someone already wrote the function. Just gotta clean it up and extend it to namespace the po files by version.

  • So the main problem I’m stuck with right now is calling add-file vs. update-file crowdin api call. They literally do only what they say.
    Guess I can’t avoid some duplication of code.

  • Along the way I refactored the version.py file where the compontents of the version are broken down into their own variables, and VERSION just reconstructing itself based on that.

  • Urgh I somehow can’t form the right API call with the requests library. Here’s the function:

    def upload_to_crowdin(files, project_key, project_id="ka-lite"):
    
        api_call = "add-file"
    
        url = "https://api.crowdin.com/api/project/{project_id}/{api_call}"
        url = url.format(project_id=project_id,
                         api_call=api_call)
    
        version_namespace = "%s.%s" % (version.MAJOR_VERSION, version.MINOR_VERSION)
    
        pot_path = pathlib.Path(POT_DIRPATH)
        files_to_upload = {"files[/KA Lite UI/%s-django.po]" % version_namespace: open((pot_path / "django.pot").resolve().__str__()),
                           "files[/KA Lite UI/%s-djangojs.po]" % version_namespace: open((pot_path / "djangojs.pot").resolve().__str__())}
        get_params = {"key": project_key}
        r = requests.post(url, params=get_params, data=files_to_upload)

    And CrowdIn is throwing me this error (as the body of r):

    '<?xml version="1.0" encoding="ISO-8859-1"?>\n<error>\n  <code>4</code>\n  <message>No files specified in request</message>\n</error>\n'
  • Finally got it working! You have to add in the file array syntax and put it in the files parameter for requests.post:

        files_to_upload = {"files[/versioned/%s-django.po]" % version_namespace: open((pot_path / "django.pot").resolve().__str__()),
                           "files[/versioned/%s-djangojs.po]" % version_namespace: open((pot_path / "djangojs.pot").resolve().__str__())}
        get_params = {"key": project_key}
        r = requests.post(url, params=get_params, files=files_to_upload)
  • Ok so the problem is that when a user clicks “Download” for a content we have no idea how long they’ve viewed it. We want to track that.

  • Jamie suggested looking into the focus/blur JS events.

{"cards":[{"_id":"48ab7b947c9342349000001b","treeId":"48765130791649108300000d","seq":756542,"position":2,"parentId":null,"content":"# MVM\n\nThe Mini VM."},{"_id":"48ab8079d606cfcaf200001c","treeId":"48765130791649108300000d","seq":621830,"position":2,"parentId":"48ab7b947c9342349000001b","content":"The beginnings"},{"_id":"48ab814fd606cfcaf200001d","treeId":"48765130791649108300000d","seq":621831,"position":1,"parentId":"48ab8079d606cfcaf200001c","content":"Picking up where i left off - I'm now writing the mark and sweep algorithm."},{"_id":"48ab822bd606cfcaf200001f","treeId":"48765130791649108300000d","seq":621832,"position":2,"parentId":"48ab8079d606cfcaf200001c","content":"Starting with the \"mark\" phase - I need to add another attribute into each object to mark it as \"reachable\"."},{"_id":"48766a0b7916491083000010","treeId":"48765130791649108300000d","seq":723717,"position":3,"parentId":null,"content":"# KA Lite Dev\n\nSome stuff in KA Lite that I devved."},{"_id":"495fa52d8ac9c87dcf000047","treeId":"48765130791649108300000d","seq":717778,"position":1.75,"parentId":"48766a0b7916491083000010","content":"# [CLOSED] ka-lite: 2408 -- add midyear exams\n\nhttps://github.com/learningequality/ka-lite/issues/2408"},{"_id":"495fa62c8ac9c87dcf000048","treeId":"48765130791649108300000d","seq":709747,"position":1,"parentId":"495fa52d8ac9c87dcf000047","content":"Hehey! This should be easy. Changes are in this commit: https://github.com/aronasorman/mlcsv/commit/8359ca3e80451b49f9215006a68bfcb632d75040"},{"_id":"495fb898ba43b68ed000002e","treeId":"48765130791649108300000d","seq":709771,"position":2,"parentId":"495fa52d8ac9c87dcf000047","content":"But hey! It's not working. It's not writing out the exam jsons. Rather than doing the normal `insert-debugger-line-here` method I normally do in python, I'm gonna try using `strace` to see what's wrong."},{"_id":"49607b60ba43b68ed000002f","treeId":"48765130791649108300000d","seq":710399,"position":3,"parentId":"495fa52d8ac9c87dcf000047","content":"I `strace`d the executable and it seems that it doesn't write the expected files at all. Which was totally what I expected :P looks I'm gonna have to use the normal debugging techniques again."},{"_id":"4960a06bba43b68ed0000030","treeId":"48765130791649108300000d","seq":710420,"position":4,"parentId":"495fa52d8ac9c87dcf000047","content":"Turns out the problem is two-fold: the Exam and playlist IDs in the new CSVs weren't in their right places, and we weren't processing lines with \"Mid Year Exam\" as an exam. Fixed those in [3c87d1c](https://github.com/aronasorman/mlcsv/commit/8359ca3e80451b49f9215006a68bfcb632d75040)"},{"_id":"496b1babba43b68ed0000031","treeId":"48765130791649108300000d","seq":797311,"position":1.875,"parentId":"48766a0b7916491083000010","content":"# [CLOSED] ka-lite: 2385: -- Integrate PDF.js into KA Lite\n\nhttps://github.com/learningequality/ka-lite/issues/2385"},{"_id":"496b1cb6ba43b68ed0000032","treeId":"48765130791649108300000d","seq":717922,"position":1,"parentId":"496b1babba43b68ed0000031","content":"OK! Goal for now is to jump off Richard's audio content html page, remove his stuff, add in PDF.js, then make it work."},{"_id":"496b1e92ba43b68ed0000033","treeId":"48765130791649108300000d","seq":717925,"position":2,"parentId":"496b1babba43b68ed0000031","content":"I've downloaded the \"build\" version of PDF.js which includes a viewer.html and some static assets. I'm looking into integrating that wholesale into KA Lite, then just ask Dylan/Jessica to integrate it."},{"_id":"496b60abba43b68ed0000034","treeId":"48765130791649108300000d","seq":718032,"position":3,"parentId":"496b1babba43b68ed0000031","content":"First step: just added `pdf.js` and `pdf.worker.js` into the static-libraries."},{"_id":"496b715bba43b68ed0000036","treeId":"48765130791649108300000d","seq":718036,"position":4,"parentId":"496b1babba43b68ed0000031","content":"I then added a \"PDF\" category into this switch statement: https://github.com/learningequality/ka-lite/pull/2399/files?diff=split#diff-df80504875a8ec892881c8b2b3d3d536R407"},{"_id":"496d686fba43b68ed0000037","treeId":"48765130791649108300000d","seq":719252,"position":5,"parentId":"496b1babba43b68ed0000031","content":"I don't have any code yet, but i'm stuck at trying to make the topic tree load a PDF. Now how would I do that? I'm looking at the JS stack trace to see how the whole thing loads."},{"_id":"496d970fba43b68ed0000038","treeId":"48765130791649108300000d","seq":719334,"position":6,"parentId":"496b1babba43b68ed0000031","content":"I just faked it by making a PDF viewer load when `kind` == \"Video\". Heh. Then just load a static pdf rather than what the actual node is. On to making the PDF viewer work!"},{"_id":"496e1263ba43b68ed0000039","treeId":"48765130791649108300000d","seq":719379,"position":7,"parentId":"496b1babba43b68ed0000031","content":"I'm considering just embedding the builtin pdf.js `viewer.html` inside an iframe for the prototype. Let's see how this goes."},{"_id":"496e719aba43b68ed000003a","treeId":"48765130791649108300000d","seq":719410,"position":8,"parentId":"496b1babba43b68ed0000031","content":"So simply embedding it in an iframe seems to be working well, except for the fact that it's too small right now. Next step is to find a way to resize it dynamically based on the window size. Bootstrap can hopefully help in that aspect."},{"_id":"496e82dcba43b68ed000003b","treeId":"48765130791649108300000d","seq":719412,"position":9,"parentId":"496b1babba43b68ed0000031","content":"And oh reminder to myself: stick `PDFJS.disableFontFace = true` into the PDF.js initialization code so that it will work on mobile browsers."},{"_id":"497af22ffdc16cdfce00003a","treeId":"48765130791649108300000d","seq":723529,"position":10,"parentId":"496b1babba43b68ed0000031","content":"OK, tried it out with the test tablet from China. Loading is a bit slow with the PDF (about 5-7 seconds) but after that there seems to be no bugs in rendering. Yahoo!"},{"_id":"497bffa1fdc16cdfce00003e","treeId":"48765130791649108300000d","seq":723845,"position":11,"parentId":"496b1babba43b68ed0000031","content":"Ok, moving to PDF progress tracking now. I'm skimming through the `viewer.js` and other files bundled by Mozilla (which I stuck inside the iframe described above). And I saw this: https://github.com/mozilla/pdf.js/blob/master/web/viewer.js#L277\n\nLooks helpful for tracking the user's progress on a PDF."},{"_id":"497c097dfdc16cdfce00003f","treeId":"48765130791649108300000d","seq":723847,"position":12,"parentId":"496b1babba43b68ed0000031","content":"I tried accessing `PDFVIew` at which `watchScroll` is defined, but it's not defined as it's named. It's probably defined somewhere as a variable. Will continue reading."},{"_id":"49ad6dbffdc16cdfce000040","treeId":"48765130791649108300000d","seq":741854,"position":13,"parentId":"496b1babba43b68ed0000031","content":"For now it seems to much work to log every page they visited, so we're going with checking if they opened the PDF for X seconds."},{"_id":"49add6cbfdc16cdfce000041","treeId":"48765130791649108300000d","seq":742134,"position":14,"parentId":"496b1babba43b68ed0000031","content":"So to do any kind of logging at all I'm gonna have to find a way to access the javascript variables inside the iFrame. Researching that now."},{"_id":"49ae3cc4fdc16cdfce000042","treeId":"48765130791649108300000d","seq":742356,"position":15,"parentId":"496b1babba43b68ed0000031","content":"So to access the `window` object of an iframe, you use `this.contentWindow`. Nifty."},{"_id":"49ae4c23fdc16cdfce000044","treeId":"48765130791649108300000d","seq":742371,"position":16,"parentId":"496b1babba43b68ed0000031","content":"Ha! So PDFJS has an event called `pagechange`: https://github.com/aronasorman/ka-lite/blob/pdfjs/static-libraries/pdfjs/web/viewer.js#L5798-L5799\n\nThere it is :D"},{"_id":"49ae4eaffdc16cdfce000045","treeId":"48765130791649108300000d","seq":742375,"position":17,"parentId":"496b1babba43b68ed0000031","content":"There's also the `DOMContentLoaded` event. Will investigate how this is used."},{"_id":"49b09bfbfdc16cdfce000047","treeId":"48765130791649108300000d","seq":743623,"position":18,"parentId":"496b1babba43b68ed0000031","content":"So I made the skeleton frontend code that listens to the `pagechange` event and the `pageshow` event to track progress. Doesn't do any actual API calls yet: https://github.com/aronasorman/ka-lite/commit/0ab9f7b6548dbd312ce8a440258bf3542240bae3\n\nI'm gonna work on tracking progress right now through the `ContentLog`."},{"_id":"49ba3bccfdc16cdfce000049","treeId":"48765130791649108300000d","seq":749954,"position":19,"parentId":"496b1babba43b68ed0000031","content":"So to fetch a ContentLog we have to ensure that the user is logged in. I just copy pasted richard's code to mine: https://github.com/aronasorman/ka-lite/blob/pdfjs/kalite/distributed/static/js/distributed/audio/views.js#L15-L30\n\nNow, this common code can be shared. I prefer a mixin, as recommended here: http://ricostacruz.com/backbone-patterns/#mixins\n\nI'll hold off on doing that though. I'll confer with Richard and Jamie on what they prefer. For now I just copy pasted that snippet."},{"_id":"49bb417bfdc16cdfce00004a","treeId":"48765130791649108300000d","seq":750833,"position":20,"parentId":"496b1babba43b68ed0000031","content":"So some of the time an event's target is the iframe's document itself. One of the difficulties I was having was how to access the `window` object from the document. To do that, I just had to call `document.defaultView` and that would return the appropriate `window` object."},{"_id":"49c767f8fdc16cdfce00004b","treeId":"48765130791649108300000d","seq":756539,"position":21,"parentId":"496b1babba43b68ed0000031","content":"Woohoo [content logging is done](https://github.com/aronasorman/ka-lite/blob/pdfjs/kalite/distributed/static/js/distributed/pdf/views.js#L71-L105)! Now on to setting points."},{"_id":"49c822b8fdc16cdfce00004d","treeId":"48765130791649108300000d","seq":758559,"position":22,"parentId":"496b1babba43b68ed0000031","content":"Point tracking done: https://github.com/aronasorman/ka-lite/commit/1f647ffd8b7b37c6caf8345d67019a1141490926\n\nSome gotchas:\n- have to update the `statusModel.points` attribute for points in the UI to update in realtime.\n- [have to edit the compute_total_points function to take the ContentLog into account](https://github.com/aronasorman/ka-lite/commit/d552ee48bc9ca1f52223cc6d00af0494cae56a2c)"},{"_id":"4b392bb60c17bf7477000055","treeId":"48765130791649108300000d","seq":917785,"position":1.984375,"parentId":"48766a0b7916491083000010","content":"# [CLOSED] ka-lite: 2548 -- Add download link for content\n\nhttps://github.com/learningequality/ka-lite/issues/2548"},{"_id":"4b392d9a0c17bf7477000056","treeId":"48765130791649108300000d","seq":917343,"position":1,"parentId":"4b392bb60c17bf7477000055","content":"Ok, the whole things has been refactored heavily by Richard. Have to read the contents of `ContentBaseView` and `BaseView`."},{"_id":"4b3a58e80c17bf7477000057","treeId":"48765130791649108300000d","seq":917784,"position":2,"parentId":"4b392bb60c17bf7477000055","content":"T'was a simple affair. Just added a link to `ContentWrapperView`'s handlebars template: https://github.com/aronasorman/ka-lite/commit/74be3d333b0b03d56a0762e3e26c7acdc3b429b1"},{"_id":"48766af77916491083000011","treeId":"48765130791649108300000d","seq":917786,"position":1.9921875,"parentId":"48766a0b7916491083000010","content":"# ka-lite: 2338\n\nhttps://github.com/learningequality/ka-lite/issues/2338\n\n2014/06/09\n\n- [ ] Write browser test\n- [ ] Make JS function that calls the proper backbone view\n- [ ] Make button in test itself that \"clears\" a test"},{"_id":"48766f4a7916491083000012","treeId":"48765130791649108300000d","seq":597196,"position":1,"parentId":"48766af77916491083000011","content":"The task is to have a \"clear test\" UI for teachers so they can retake it."},{"_id":"487670857916491083000013","treeId":"48765130791649108300000d","seq":597197,"position":2,"parentId":"48766af77916491083000011","content":"Backend wise, seems like the easiest way is to just delete the test log, then make an endpoint for it, and finally a button."},{"_id":"48774e117916491083000015","treeId":"48765130791649108300000d","seq":597322,"position":3,"parentId":"48766af77916491083000011","content":"Lots of refactoring work for the StudentTestTest. Need to update it to use mixins."},{"_id":"48774fd17916491083000016","treeId":"48765130791649108300000d","seq":597324,"position":4,"parentId":"48766af77916491083000011","content":"As well as update it to use the new Test.total_questions attribute.\n"},{"_id":"4877528e7916491083000018","treeId":"48765130791649108300000d","seq":597325,"position":5,"parentId":"48766af77916491083000011","content":"Now apparently some old call of `self.create_facility` is failing. Due to changed forms again?"},{"_id":"487767577916491083000019","treeId":"48765130791649108300000d","seq":597329,"position":6,"parentId":"48766af77916491083000011","content":"Specifically, this line: `https://github.com/aronasorman/ka-lite/blob/nalanda-rct3/kalite/distributed/tests/browser_tests/base.py`\n\nWith this error:\n```\nTraceback (most recent call last):\n File \"/home/aron/src/ka-lite/kalite/distributed/tests/browser_tests/student_testing.py\", line 44, in setUp\n super(StudentTestTest, self).setUp()\n File \"/home/aron/src/ka-lite/kalite/distributed/tests/browser_tests/base.py\", line 207, in setUp\n self.facility = self.create_facility(facility_name=self.facility_name)\n File \"/home/aron/src/ka-lite/kalite/testing/mixins/facility_mixins.py\", line 17, in create_facility\n obj, created = Facility.objects.get_or_create(**fields)\n File \"/home/aron/src/ka-lite/python-packages/django/db/models/manager.py\", line 146, in get_or_create\n return self.get_query_set().get_or_create(**kwargs)\n File \"/home/aron/src/ka-lite/python-packages/django/db/models/query.py\", line 470, in get_or_create\n return self.get(**lookup), False\n File \"/home/aron/src/ka-lite/python-packages/django/db/models/query.py\", line 379, in get\n clone = self.filter(*args, **kwargs)\n File \"/home/aron/src/ka-lite/python-packages/django/db/models/query.py\", line 655, in filter\n return self._filter_or_exclude(False, *args, **kwargs)\n File \"/home/aron/src/ka-lite/python-packages/django/db/models/query.py\", line 673, in _filter_or_exclude\n clone.query.add_q(Q(*args, **kwargs))\n File \"/home/aron/src/ka-lite/python-packages/django/db/models/sql/query.py\", line 1266, in add_q\n can_reuse=used_aliases, force_having=force_having)\n File \"/home/aron/src/ka-lite/python-packages/django/db/models/sql/query.py\", line 1134, in add_filter\n process_extras=process_extras)\n File \"/home/aron/src/ka-lite/python-packages/django/db/models/sql/query.py\", line 1332, in setup_joins\n \"Choices are: %s\" % (name, \", \".join(names)))\nFieldError: Cannot resolve keyword 'facility_name' into field. Choices are: address, address_normalized, contact_email, contact_name, contact_phone, counter, deleted, description, facilitygroup, facilityuser, id, latitude, longitude, name, signature, signed_by, signed_version, user_count, zone_fallback, zoom\n```"},{"_id":"48776c16791649108300001a","treeId":"48765130791649108300000d","seq":597330,"position":7,"parentId":"48766af77916491083000011","content":"Just removed that line. Heh."},{"_id":"48777065791649108300001b","treeId":"48765130791649108300000d","seq":597331,"position":8,"parentId":"48766af77916491083000011","content":"Ok, I seem to be getting an unrelated Selenium error, with Firefox crashing with this message:\n```\nTraceback (most recent call last):\n File \"/home/aron/src/ka-lite/kalite/testing/browser.py\", line 174, in setUp\n (self.browser, self.admin_user, self.admin_pass) = setup_test_env(browser_type=browser_type)\n File \"/home/aron/src/ka-lite/kalite/testing/browser.py\", line 38, in setup_test_env\n local_browser = getattr(webdriver, browser_type)() # Get local session of browser\n File \"/home/aron/src/ka-lite/python-packages/selenium/webdriver/firefox/webdriver.py\", line 59, in __init__\n self.binary, timeout),\n File \"/home/aron/src/ka-lite/python-packages/selenium/webdriver/firefox/extension_connection.py\", line 47, in __init__\n self.binary.launch_browser(self.profile)\n File \"/home/aron/src/ka-lite/python-packages/selenium/webdriver/firefox/firefox_binary.py\", line 61, in launch_browser\n self._wait_until_connectable()\n File \"/home/aron/src/ka-lite/python-packages/selenium/webdriver/firefox/firefox_binary.py\", line 105, in _wait_until_connectable\n self.profile.path, self._get_firefox_output()))\nWebDriverException: Message: 'Can\\'t load the profile. Profile Dir: /tmp/tmpbXa7eJ Firefox output: \\n(process:27038): GLib-CRITICAL **: g_slice_set_config: assertion \\'sys_page_size == 0\\' failed\\n1410068812152\\taddons.manager\\tDEBUG\\tLoaded provider scope for resource://gre/modules/addons/XPIProvider.jsm: [\"XPIProvider\"]\\n1410068812154\\taddons.manager\\tDEBUG\\tLoaded provider scope for resource://gre/modules/LightweightThemeManager.jsm: [\"LightweightThemeManager\"]\\n1410068812156\\taddons.xpi\\tDEBUG\\tstartup\\n1410068812157\\taddons.xpi\\tINFO\\tMapping online-accounts@lists.launchpad.net to /usr/lib/mozilla/extensions/{ec8030f7-c20a-464f-9b0e-13a3a9e97384}/online-accounts@lists.launchpad.net\\n1410068812158\\taddons.xpi\\tINFO\\tMapping {2e1445b0-2682-11e1-bfc2-0800200c9a66} to /usr/share/mozilla/extensions/{ec8030f7-c20a-464f-9b0e-13a3a9e97384}/{2e1445b0-2682-11e1-bfc2-0800200c9a66}\\n1410068812158\\taddons.xpi\\tINFO\\tMapping webapps-team@lists.launchpad.net to /usr/share/mozilla/extensions/{ec8030f7-c20a-464f-9b0e-13a3a9e97384}/webapps-team@lists.launchpad.net\\n1410068812158\\taddons.xpi\\tINFO\\tMapping ubufox@ubuntu.com to /usr/share/mozilla/extensions/{ec8030f7-c20a-464f-9b0e-13a3a9e97384}/ubufox@ubuntu.com\\n1410068812158\\taddons.xpi\\tINFO\\tMapping {972ce4c6-7e08-4474-a285-3208198ce6fd} to /usr/lib/firefox/browser/extensions/{972ce4c6-7e08-4474-a285-3208198ce6fd}\\n1410068812158\\taddons.xpi\\tINFO\\tMapping langpack-en-ZA@firefox.mozilla.org to /usr/lib/firefox/browser/extensions/langpack-en-ZA@firefox.mozilla.org.xpi\\n1410068812159\\taddons.xpi\\tINFO\\tMapping langpack-en-GB@firefox.mozilla.org to /usr/lib/firefox/browser/extensions/langpack-en-GB@firefox.mozilla.org.xpi\\n1410068812159\\taddons.xpi\\tINFO\\tMapping fxdriver@googlecode.com to /tmp/tmpbXa7eJ/extensions/fxdriver@googlecode.com\\n1410068812160\\taddons.xpi\\tDEBUG\\tcheckForChanges\\n1410068812171\\taddons.xpi\\tDEBUG\\tNo changes found\\n1410068812176\\taddons.xpi\\tDEBUG\\tRegistering manifest for /usr/lib/firefox/browser/extensions/langpack-en-ZA@firefox.mozilla.org.xpi\\n1410068812177\\taddons.xpi\\tDEBUG\\tRegistering manifest for /usr/lib/firefox/browser/extensions/langpack-en-GB@firefox.mozilla.org.xpi\\n*** Blocklist::_preloadBlocklistFile: blocklist is disabled\\n' \n```\n\nUgh. What will I do to make this test run?"},{"_id":"48778213791649108300001c","treeId":"48765130791649108300000d","seq":597333,"position":9,"parentId":"48766af77916491083000011","content":"Debugging that firefox error later. Dropped back to phantomjs. Getting this error now:\n```\nERROR: test_exercise_mastery (kalite.distributed.tests.browser_tests.student_testing.StudentTestTest)\n----------------------------------------------------------------------\nTraceback (most recent call last):\n File \"/home/aron/src/ka-lite/kalite/distributed/tests/browser_tests/student_testing.py\", line 46, in setUp\n self.fac = self.create_facility(self.facility_name)\nTypeError: create_facility() takes exactly 1 argument (2 given)\n```\n\nSeems like we're still getting the old \"create_facility\" defined in FacilityTestCase. How to skip that?"},{"_id":"48778cbb791649108300001d","treeId":"48765130791649108300000d","seq":597338,"position":10,"parentId":"48766af77916491083000011","content":"Turns out `create_facility` only accepts KEYWORD arguments, and we're passing the facility as a POSITIONAL argument. The correct line should be:\n```python\n self.fac = self.create_facility(name=self.facility_name)\n```"},{"_id":"48778efe791649108300001e","treeId":"48765130791649108300000d","seq":597341,"position":11,"parentId":"48766af77916491083000011","content":"Now we're getting a timeout:\n```\n======================================================================\nERROR: test_exercise_mastery (kalite.distributed.tests.browser_tests.student_testing.StudentTestTest)\n----------------------------------------------------------------------\nTraceback (most recent call last):\n File \"/home/aron/src/ka-lite/kalite/distributed/tests/browser_tests/student_testing.py\", line 85, in test_exercise_mastery\n self.browser_submit_answers(answer)\n File \"/home/aron/src/ka-lite/kalite/distributed/tests/browser_tests/student_testing.py\", line 67, in browser_submit_answers\n (By.CSS_SELECTOR, \"#solutionarea > input[type=text]\")))\n File \"/home/aron/src/ka-lite/python-packages/selenium/webdriver/support/wait.py\", line 71, in until\n raise TimeoutException(message)\nTimeoutException: Message: '' \n\n```"},{"_id":"4877a7779fe4725d1e000012","treeId":"48765130791649108300000d","seq":597345,"position":12,"parentId":"48766af77916491083000011","content":"It's failing with some UI exception, but I can't see the error because I'm running phantomjs. Time to make Firefox work."},{"_id":"4877b7669fe4725d1e000013","treeId":"48765130791649108300000d","seq":597409,"position":13,"parentId":"48766af77916491083000011","content":"as per this stack overflow answer: http://stackoverflow.com/a/25645344\n\nI had to downgrade my version of Firefox to version 30 to make everything work again."},{"_id":"487804e89fe4725d1e000014","treeId":"48765130791649108300000d","seq":597411,"position":14,"parentId":"48766af77916491083000011","content":"Ran the `test_exercise_mastery` test again. Success."},{"_id":"487805ec9fe4725d1e000015","treeId":"48765130791649108300000d","seq":597413,"position":15,"parentId":"48766af77916491083000011","content":"Ran all tests. Failure, with half of them coming from the deletion of the default facility for KALiteDistributedBrowserTestCase. Fixing."},{"_id":"4a25ac8531e93924a100004d","treeId":"48765130791649108300000d","seq":917788,"position":1.99609375,"parentId":"48766a0b7916491083000010","content":"# [CLOSED] ka-lite: 2455 -- Extend search functionality to include other content\n\nhttps://github.com/learningequality/ka-lite/issues/2455"},{"_id":"4a25ae0831e93924a100004e","treeId":"48765130791649108300000d","seq":798620,"position":1,"parentId":"4a25ac8531e93924a100004d","content":"Ok, first off develop doesn't have a search bar anymore. So I added it back with this code:\n```html\n{# Search box #}\n <li>\n <form class=\"navbar-form\" action=\"{% url 'search' %}\" method=\"get\" id=\"search-box\" role=\"search\">\n {% comment %}translators: this will appear in the navigation bar. please try to keep it as short as possible.{% endcomment %}\n <div class=\"input-group\">\n <input type=\"text\" name=\"query\" id=\"search\" class=\"form-control\" placeholder=\"{% trans 'video, topic, or exercise...' %}\" />\n <button class=\"btn btn-default\" type=\"submit\"><i class=\"glyphicon glyphicon-search\"></i></button>\n </div>\n </form>\n </li>\n```"},{"_id":"4a25b0f631e93924a100004f","treeId":"48765130791649108300000d","seq":798626,"position":2,"parentId":"4a25ac8531e93924a100004d","content":"Second, I'm gonna have to make the JS code in `search_autocomplete.js` at least find that search box so I can see what endpoint is being called."},{"_id":"4a25b6f731e93924a1000050","treeId":"48765130791649108300000d","seq":798645,"position":3,"parentId":"4a25ac8531e93924a100004d","content":"NVM. Turns out it's already detecting it once I put a breakpoint on `fetchTopicTree`. Now I gotta extend that API endpoint to return other content types as well."},{"_id":"4a25bb8931e93924a1000051","treeId":"48765130791649108300000d","seq":798716,"position":4,"parentId":"4a25ac8531e93924a100004d","content":"Querying the api directly on the browser (with `http://127.0.0.1:8008/api/flat_topic_tree/en`) resulted in this JSON:\n\n```\n{\nTopic: {\ndemo: {\navailable: true,\npath: \"demo/\",\nkind: \"Topic\",\ntitle: \"demo\"\n},\nclassrooms: {\navailable: true,\npath: \"demo/classrooms/\",\nkind: \"Topic\",\ntitle: \"classrooms\"\n},\nphysics: {\navailable: true,\npath: \"demo/science/physics/\",\nkind: \"Topic\",\ntitle: \"physics\"\n},\nmath: {\navailable: true,\npath: \"demo/math/\",\nkind: \"Topic\",\ntitle: \"math\"\n},\nscience: {\navailable: true,\npath: \"demo/science/\",\nkind: \"Topic\",\ntitle: \"science\"\n}\n},\nVideo: {\n3db321ccaec3154139dd42ded78f19b5: {\navailable: true,\npath: \"demo/classrooms/healing-classrooms-1\",\nkind: \"Video\",\ntitle: \"Healing Classrooms 1\"\n}\n},\nExercise: { }\n}\n```\n\nSo it's mostly working already, I just need to extend it to add the other content types. Whoo! Thought I need to do more hackzor.\n"},{"_id":"4a25c03a31e93924a1000052","treeId":"48765130791649108300000d","seq":799119,"position":5,"parentId":"4a25ac8531e93924a100004d","content":"Alright so I actually had to edit `generate_node_cache` to add in other types, like so: https://github.com/aronasorman/ka-lite/blob/search/kalite/topic_tools/__init__.py#L259-L260\n\nI also had to edit `get_content_cache` to be able to filter the content: https://github.com/aronasorman/ka-lite/blob/search/kalite/topic_tools/__init__.py#L113-L126\n\nSo now the API endpoint returns the content kinds we explicitly support! Nice!"},{"_id":"4a26a23531e93924a1000053","treeId":"48765130791649108300000d","seq":823665,"position":6,"parentId":"4a25ac8531e93924a100004d","content":"So I really didn't do much, and the search autocomplete is now working already. Problems I saw:\n\n- No icons for the new content types\n- The separate search page isn't styled correctly\n- Pressing enter for text like \"math\" leads to a 404."},{"_id":"4b3a5b550c17bf7477000058","treeId":"48765130791649108300000d","seq":917789,"position":7,"parentId":"4a25ac8531e93924a100004d","content":"Passed off to Richard."},{"_id":"48a85b719a5dd8d731000021","treeId":"48765130791649108300000d","seq":917790,"position":1.998046875,"parentId":"48766a0b7916491083000010","content":"# ka-lite: 2238 -- update crowdin management command\n\nhttps://github.com/learningequality/ka-lite/issues/2238\n"},{"_id":"48a901639a5dd8d731000022","treeId":"48765130791649108300000d","seq":620270,"position":1,"parentId":"48a85b719a5dd8d731000021","content":"So first step is to make a skeleton management command. I used `NoArgsCommand` since I don't expect any argument to this."},{"_id":"48a907f19a5dd8d731000023","treeId":"48765130791649108300000d","seq":620271,"position":2,"parentId":"48a85b719a5dd8d731000021","content":"Proceeding to make a test for this. The basic aim of the test is to see if we're making a request to the right url. So it will involve the `mock` library."},{"_id":"48a957559a5dd8d731000024","treeId":"48765130791649108300000d","seq":620354,"position":3,"parentId":"48a85b719a5dd8d731000021","content":"Mocking requests might be easier with [httreplay](https://github.com/davepeck/httreplay). Looking into it to see if it's easy to work with."},{"_id":"48b29a409a5dd8d73100002b","treeId":"48765130791649108300000d","seq":626085,"position":4,"parentId":"48a85b719a5dd8d731000021","content":"I saw this cool library called [vcr.py](https://github.com/kevin1024/vcrpy). Kinda like `vcr` from the Rails community. I'm thinking of switching to this from manually constructing request mock objects as it will be easier to keep up to date with the remote server."},{"_id":"48b29d999a5dd8d73100002c","treeId":"48765130791649108300000d","seq":626088,"position":5,"parentId":"48a85b719a5dd8d731000021","content":"I'm thinking of making a wrapper for `vcr.py` so that testers don't have to fudge with the cassette names. They just call a `with` statement (or a decorator) and boom!"},{"_id":"48b2a4089a5dd8d73100002d","treeId":"48765130791649108300000d","seq":626095,"position":6,"parentId":"48a85b719a5dd8d731000021","content":"The wrapper will be the new KALiteTestCase. Lelz."},{"_id":"48b2f7429a5dd8d73100002e","treeId":"48765130791649108300000d","seq":626368,"position":7,"parentId":"48a85b719a5dd8d731000021","content":"I had KALiteTestCase inherit from `CreateDeviceMixin` and `TestCase`. That caused a couple of MRO resolution errors, which I promptly fixed. From `KALiteTestCase` I made a `KALiteClientTestCase` which is for Django client tests, and a `KALiteBrowserTestCase` which is for browser tests.\n\nI'm encountering some test failures now with `student_testing.CoreTests`, which I modified to inherit from `KALiteClientTestCase`. The errors look something like:\n\n```\nTraceback (most recent call last):\n File \"/home/aron/src/ka-lite/kalite/student_testing/tests.py\", line 81, in setUp\n super(CoreTests, self).setUp()\n File \"/home/aron/src/ka-lite/kalite/student_testing/tests.py\", line 38, in setUp\n self.assertTrue(self.client.facility)\nAssertionError: None is not True\n```"},{"_id":"48b30b3c9a5dd8d73100002f","treeId":"48765130791649108300000d","seq":626406,"position":8,"parentId":"48a85b719a5dd8d731000021","content":"Fixed. Just had `CoreTests` and `CurrentUnitTests` inherit from `KALiteClientTestCase`."},{"_id":"48b4deed2552bf07b100002e","treeId":"48765130791649108300000d","seq":627299,"position":9,"parentId":"48a85b719a5dd8d731000021","content":"Now fixing more test breakage caused by MRO errors (a test case inheriting from both `CreateDeviceMixin` and `KALiteTestCase`, which itself inherits `CreateDeviceMixin`)."},{"_id":"48b4e6f32552bf07b100002f","treeId":"48765130791649108300000d","seq":627310,"position":10,"parentId":"48a85b719a5dd8d731000021","content":"Now getting this error when running all the tests:\n\n```\nTraceback (most recent call last):\n File \"/home/aron/src/ka-lite/python-packages/django/core/management/base.py\", line 222, in run_from_argv\n self.execute(*args, **options.__dict__)\n File \"/home/aron/src/ka-lite/python-packages/django/core/management/commands/test.py\", line 80, in execute\n super(Command, self).execute(*args, **options)\n File \"/home/aron/src/ka-lite/python-packages/django/core/management/base.py\", line 255, in execute\n output = self.handle(*args, **options)\n File \"/home/aron/src/ka-lite/python-packages/south/management/commands/test.py\", line 8, in handle\n super(Command, self).handle(*args, **kwargs)\n File \"/home/aron/src/ka-lite/python-packages/django/core/management/commands/test.py\", line 106, in handle\n failures = test_runner.run_tests(test_labels)\n File \"/home/aron/src/ka-lite/kalite/testing/testrunner.py\", line 68, in run_tests\n return run_tests_wrapper_fn()\n File \"/home/aron/src/ka-lite/python-packages/django/test/utils.py\", line 220, in inner\n return test_func(*args, **kwargs)\n File \"/home/aron/src/ka-lite/kalite/testing/testrunner.py\", line 67, in run_tests_wrapper_fn\n return super(KALiteTestRunner,self).run_tests(test_labels, extra_tests, **kwargs)\n File \"/home/aron/src/ka-lite/python-packages/django/test/simple.py\", line 366, in run_tests\n suite = self.build_suite(test_labels, extra_tests)\n File \"/home/aron/src/ka-lite/kalite/testing/testrunner.py\", line 79, in build_suite\n testfun = getattr(test, test._testMethodName)\nAttributeError: 'TestSuite' object has no attribute '_testMethodName'\n```\n\nInvestigating."},{"_id":"48b4f0642552bf07b1000030","treeId":"48765130791649108300000d","seq":627336,"position":11,"parentId":"48a85b719a5dd8d731000021","content":"Only happens on the code called when `--failfast` is on. Something related to these lines, found in:\n```python\n if self.failfast:\n for test in test_suite._tests:\n testfun = getattr(test, test._testMethodName)\n setattr(test, test._testMethodName, auto_pdb()(testfun))\n return test_suite\n```"},{"_id":"48b5b5132552bf07b1000031","treeId":"48765130791649108300000d","seq":627557,"position":12,"parentId":"48a85b719a5dd8d731000021","content":"....\nI got into this refactoring blackhole. Coming back out with refactored tests!"},{"_id":"48c0476c2552bf07b1000032","treeId":"48765130791649108300000d","seq":632800,"position":13,"parentId":"48a85b719a5dd8d731000021","content":"While rewriting the tests, I decided it would be nice to have a multiple cursor feature like sublime text. So I just downloaded the `multiple-cursors` library and integrated it into Emacs."},{"_id":"495f8e378ac9c87dcf000046","treeId":"48765130791649108300000d","seq":709682,"position":14,"parentId":"48a85b719a5dd8d731000021","content":"Back! Turns out [someone already wrote the function](https://github.com/fle-internal/ka-lite-central/blob/7afcc6f27977b906251d63c2bbb3e48c937af4cd/centralserver/i18n/management/commands/update_pot.py#L121). Just gotta clean it up and extend it to namespace the po files by version."},{"_id":"49c89783fdc16cdfce00004e","treeId":"48765130791649108300000d","seq":758731,"position":15,"parentId":"48a85b719a5dd8d731000021","content":"So the main problem I'm stuck with right now is calling `add-file` vs. `update-file` crowdin api call. They literally do only what they say.\n Guess I can't avoid some duplication of code."},{"_id":"49d5934efdc16cdfce000050","treeId":"48765130791649108300000d","seq":766546,"position":16,"parentId":"48a85b719a5dd8d731000021","content":"Along the way [I refactored the `version.py`](https://github.com/learningequality/ka-lite/commit/db4975a8ea00f5e2d292e7defbc093370e2be722) file where the compontents of the version are broken down into their own variables, and VERSION just reconstructing itself based on that."},{"_id":"49d72bbdb593ad075b00004b","treeId":"48765130791649108300000d","seq":766556,"position":17,"parentId":"48a85b719a5dd8d731000021","content":"Urgh I somehow can't form the right API call with the `requests` library. Here's the function:\n```python\ndef upload_to_crowdin(files, project_key, project_id=\"ka-lite\"):\n\n api_call = \"add-file\"\n\n url = \"https://api.crowdin.com/api/project/{project_id}/{api_call}\"\n url = url.format(project_id=project_id,\n api_call=api_call)\n\n version_namespace = \"%s.%s\" % (version.MAJOR_VERSION, version.MINOR_VERSION)\n\n pot_path = pathlib.Path(POT_DIRPATH)\n files_to_upload = {\"files[/KA Lite UI/%s-django.po]\" % version_namespace: open((pot_path / \"django.pot\").resolve().__str__()),\n \"files[/KA Lite UI/%s-djangojs.po]\" % version_namespace: open((pot_path / \"djangojs.pot\").resolve().__str__())}\n get_params = {\"key\": project_key}\n r = requests.post(url, params=get_params, data=files_to_upload)\n```\n\nAnd CrowdIn is throwing me this error (as the body of `r`):\n```\n'<?xml version=\"1.0\" encoding=\"ISO-8859-1\"?>\\n<error>\\n <code>4</code>\\n <message>No files specified in request</message>\\n</error>\\n'\n```"},{"_id":"49d80b6db593ad075b00004c","treeId":"48765130791649108300000d","seq":766680,"position":18,"parentId":"48a85b719a5dd8d731000021","content":"Finally got it working! You have to add in the file array syntax **and** put it in the files parameter for `requests.post`:\n```python\n files_to_upload = {\"files[/versioned/%s-django.po]\" % version_namespace: open((pot_path / \"django.pot\").resolve().__str__()),\n \"files[/versioned/%s-djangojs.po]\" % version_namespace: open((pot_path / \"djangojs.pot\").resolve().__str__())}\n get_params = {\"key\": project_key}\n r = requests.post(url, params=get_params, files=files_to_upload)\n```"},{"_id":"4b44e5b70c17bf7477000059","treeId":"48765130791649108300000d","seq":920234,"position":1.9990234375,"parentId":"48766a0b7916491083000010","content":"# ka-lite 2566 -- Track downloaded content\n\nhttps://github.com/learningequality/ka-lite/issues/2566"},{"_id":"4b44e6a90c17bf747700005a","treeId":"48765130791649108300000d","seq":920238,"position":1,"parentId":"4b44e5b70c17bf7477000059","content":"Ok so the problem is that when a user clicks \"Download\" for a content we have no idea how long they've viewed it. We want to track that."},{"_id":"4b44ea7d0c17bf747700005b","treeId":"48765130791649108300000d","seq":920239,"position":2,"parentId":"4b44e5b70c17bf7477000059","content":"Jamie suggested looking into the focus/blur JS events."}],"tree":{"_id":"48765130791649108300000d","name":"Aron's Dev Journal","publicUrl":"arons-dev-journal"}}