posted by seamus5 on July 10, 2012 | 38 comments
Hey guys, I have been noticing recently that when I do reviews, this same exact question pops up very frequently. This happens especially after I answer a question incorrectly. I know this is probably related to SRS function, but it seems to be much more frequent than usual.

so for example, I got the opinion for a hanzi wrong. The next four questions, were related to that particular character. The next question was what is the English, then what is the English, then what is that pinyin.

I have also noticed that when I am doing larger reviews, with many words in them, this still happens but these words are spaced throughout the entire review process more.

I was kind of curious if this high level of frequency was done intentionally? It would make sense to have higher frequency for something that you may be getting incorrect, but I am wondering if you also intended it to be this frequent as well?

One of the problems that I have been noticing though, is that it will repeat the exact same question two or three times in a row. So I did not know what charactered just now, and got it incorrect, but the next three questions work exactly the same as the one that I had before. This also did not help me to learn what the pinyin for that character was.

Keep up the great work!
signin to comment
bretter23 on July 11, 2012 | reply
Me too!

seamus5 on July 11, 2012 | reply
Also, repeating the missed word is fine, but I think it would be better if how you studied that word varied more. So instead of always having to select the translation after seeing the characters, maybe type the english definition once, select the correct pinyin from a list once, write the correct pinyin, fill in a sentence with the word, show the english and have to select the correct character (I don't think this happens at all in the reviews. I might need to check my settings...)
trevelyan on July 12, 2012 | reply

OK... I've just spent a few hours mucking around with things on the backend to try and improve our default settings. This has resulted in a few changes that I think should improve the quality of the test and eliminate some of these frustrations. Specifically:

(1) previously we required people to answer words correctly 3 times before the words would enter delayed SRS testing. This could result in someone getting tested on the same word three times in succession before it would go away. To avoid this we've just reduced the number of forced initial tests to 2 times. So words will enter delayed testing faster now.

(2) the order in which we were pulling content from our database for testing was also not properly randomized, which meant that newly-added vocabulary was far more likely to be tested than older words. We want to have a slight bias towards newer materials since we can't rely on everyone to keep their SRS lists clean, but I've added a probabilistic check which will do a better job scrambling test options the majority of the time. So there is still a bias towards new materials, but not an overpowering one.

I'm also taking a look at tweaking our algorithms to enforce more variety in the initial testing circuit. Since we tend to default to English language tests, one idea is defaulting to a pinyin-test as our second test of word recognition assuming user settings permit. I do not want to add forced generative testing since it is much harder for people using mobile devices to quickly review content using those test options.

seamus5 on July 12, 2012 | reply
Hey David! That sounds great. I am looking forward to giving it a try in about a hour. Thank you very much!.
jasper.obviar on July 16, 2012 | reply
Hi David,

Been having the same repeating question since yesterday. I have been answering the same question about 皮搋子 over and over again for up to 5 times; and then a different question will come up once; and then the same repeating question about 皮搋子 comes up again 5 times and so on. My SRS settings are (SRS Rank -> Test Delay):

0 -> none

1 -> 5

2 -> 12

3 -> 32

4 -> 48

5 -> 56

6 -> 64

7 -> 96
trevelyan on July 16, 2012 | reply
Update: I've taken a look and have sent you an email. It does not look like this is a major problem with the system, although it does seem that it is possible to have multiple correct answers in some cases, and this is something that we'll need to fix.
seamus5 on July 23, 2012 | reply
Hey David, just saw your reply to this, and I am assuming that this is the same as my issue. I have been answering 打过去 question every 2nd or 3rd round. The interesting thing though is that it is almost exclusively the 'write the correct pinyin' question, or 'here is the hanzi, select the correct translation' question.
trevelyan on July 25, 2012 | reply
Jasper has reported the same problems. Doubtless something I've mucked up on the backend. Will take a look this evening and hopefully have a better answer for you tomorrow.
LincolnDog on July 25, 2012 | reply
I have the same problem where one word repeats, perhaps 20 times in a group of 100 words. I got so sick of the one word, I deleted it from my vocab list, but then another word just took its place!
trevelyan on July 25, 2012 | reply
Hi Everyone,

Yes -- it was an issue on our side. The system wasn't properly tagging correct answers, and a bias we had introduced to try to privilege new words was resulting in the same word being randomly suggested 20 percent of the time. So correct answers weren't shuffled away and the same questions could be asked repeatedly in short sequence.

Both of these issues have been fixed. I'm afraid that you'll still have to answer each word correctly once more to shuffle it off into SRS-land, but hopefully this will end the frustration that people have been having, with my apologies for causing them in the first place and thanks for the help debugging this too.


drummerboy on July 26, 2012 | reply
Hi David,

Thanks for continuing to make the site better. When you have a chance can you look into a couple of issues for me? The first is that the pin yin no longer displays in review mode. The second is that the review mode of the lesson vocabulary within each lesson is not working properly. The last one is that I'm not able to download individual lessons from my iPad and/or iPod by using the link on your site. Whehn I try it tells me that "safari cannot open this page because the link is invalid. The only way I can do it is to go into iTunes. The problem with that is that not all the lessons are displayed. For example, I am working on the 红楼梦 lessons and would love to listen to them in the way to work, but I can't download them.

Thanks :-)
drummerboy on July 26, 2012 | reply
Hi David,

Thanks for the speedy fix on those first 2 items :-)
LincolnDog on July 25, 2012 | reply
David, thanks so much for sticking with this. I would thank you 20 times in a row, but you get the idea!

LincolnDog on July 28, 2012 | reply
Is anyone else having problems with review words including vocabulary that is not from studied lessons? This only seems to happen when "all" is picked, not when I limit myself to specific lessons.