Bad default choices for Vietnamese installation

Bug #191451 reported by Vu Do Quynh
18
Affects Status Importance Assigned to Milestone
console-setup (Ubuntu)
Fix Released
Undecided
Colin Watson
Declined for Hardy by Daniel Holbach
gtk+2.0 (Ubuntu)
Fix Released
Medium
Ubuntu Desktop Bugs
Declined for Hardy by Daniel Holbach
m17n-db (Ubuntu)
Fix Released
Undecided
Unassigned
Declined for Hardy by Daniel Holbach

Bug Description

1) Using live CD Desktop i386 Ubuntu 7.10 Gutsy Gibbon (official one, from shipit) ;

2) From the beginning screen :
Pressing F2: Choosing Vietnamese ;
Pressing F3: keyboard Vietnam is selected by default ! Although this choice seems logic, but the Vietnam keyboard layout does not reflect the actual situation in Vietnam where almost all keyboards are of the US English type. Leaving Vietnam keyboards result in odd keys outputs that may create confusion.

3) Booting the computer with the live CD (Language Vietnamese, Keyboard US English) without problems.

4) Main bug description:

SCIM by default is configured for Vietnamese typing. But there is only one choice of Vietnamese character input mode : Viqr, which is activated by default.

The result is that when you enter any vowel followed by a point, it results in a Vietnamese accentuated character which could leads to misinterpretation and confusion (especially with typing passwords issues).

Exemple, after opening Mozilla Firefox or a Terminal, if you would type the following 5 characters "asie.", it will results in the following 4 characters "asiẹ". Then, you would need to delete the "ẹ" and do the following typing "asie", followed by pressing the right-arrow key to move forward, then pressing the "." key to get "asie.". This looks rather clumsy to me.

Comments: Viqr input mode is common among Vietnamese living out from the country. Vietnamese people living in Vietnam have other choices, mainly Telex and/or VNI input modes. The official choice from the Vietnamese government is the Telex input mode. I think that including other Vietnamese character input modes (Telex and VNI) and setting the default input mode to Telex would be appropriate. The least effort to do would be to deactivate the Viqr input mode by default.

Revision history for this message
Vu Do Quynh (vu-do-quynh) wrote :
Revision history for this message
David Tremblay (david-tremblay) wrote :

The VNI input and the telex input can be installed with the scim-m17n package. But I second this as 99% of mainland vietnamese would expect a straight US keyboard out of the box

Revision history for this message
SK (stephantom) wrote :

Thank you for taking the time to report this issue and helping to improve Ubuntu!
This report was not reported against a specific package. I have assigned it to the Ubuntu installer package now as it seems to be the best guess. Correct it if I am wrong!
Please try to file future bug reports against the right packages. You can learn to find the right package at https://wiki.ubuntu.com/Bugs/FindRightPackage
I am also confirming this as there are multiple sources requesting that change. Maybe this should also be changed to priority:'wishlist'.
Feel free to report any other issues you might find!

Revision history for this message
Vu Do Quynh (vu-do-quynh) wrote :

Thank you for your comment. I did try to use FindRightPackage but I was unsure about which package to choose, weither it should be Ubiquity or Scim itself.
I feel that setting the importance to only "wishlist" seems too low for me (that's from a Vietnamese point of view, of course).

As I'm not a real programmer, I don't know how much complicated it is to do modifications, e.g. search in Ubiquity to deactivate Viqr in SCIM whenever Vietnamese is chosen for installation or adding the m17n package, but here also then there should be a prority to e.g. Telex mode input rather than Viqr's one.

Revision history for this message
David Tremblay (david-tremblay) wrote :

That is definitively not a wishlist. Vietnamese who install ubuntu will not be able to login after install. Using VIQR input method at login screen or any input method other than the standard US keyboard is a no go

As for the package, it is related to the input method use in GDM,

Revision history for this message
Colin Watson (cjwatson) wrote :

Should it be us(intl) - that is, "USA - International (with dead keys)", or just the standard us layout? (I've created a console-setup task for that portion of the bug.)

I'm not sure what to do with the other part. The console-setup change should automatically cause the Viqr input method no longer to be used, but doesn't really help you if you actually wanted the Telex input method. Do you know if that input method is packaged anywhere in Ubuntu?

Changed in console-setup:
assignee: nobody → kamion
status: New → Incomplete
Revision history for this message
Vu Do Quynh (vu-do-quynh) wrote :

Hi,
Yes I think that "Standard US Layout" would be fine!
At least, it's the choice that I currently do with all Ubuntu installations I've made using Qwerty keyboards in Vietnam so far.

With regards to the Telex input, I'm not very sure what to say. I know that there is a "m17n" package which adds a lot of input types for a lot of languages and that Telex is included.

By now, starting with Gutsy, we do a standard installation (either En US, or vi VN) and then we install the "hanoilug-desktop" after adding the source from Hanoilug. Details (Hanoilug repository and GPG signature key) are here: http://www.hanoilug.org/dokuwiki/soft:apt:hanoilug

Revision history for this message
David Tremblay (david-tremblay) wrote : Re: [Bug 191451] Re: Bad default choices for Vietnamese installation

> I'm not sure what to do with the other part. The console-setup change
> should automatically cause the Viqr input method no longer to be used,
> but doesn't really help you if you actually wanted the Telex input
> method. Do you know if that input method is packaged anywhere in Ubuntu?

I think this issue belong elsewhere, m17n should only be a default when
installing the vi-vn langpack so to make the telex and the vni method
available after login and and default . I'll go to see the package
itself. GDM Login must use "Standard US Layout" no doubt

> ** Also affects: console-setup (Ubuntu)
> Importance: Undecided
> Status: New
>
> ** Changed in: console-setup (Ubuntu)
> Assignee: (unassigned) => Colin Watson (kamion)
> Status: New => Incomplete
>

Revision history for this message
Jean Christophe André (progfou) wrote :

Ok... Let's precise a few points! ;-)

First of all this bug report really should have been split into two or even three ones (not sure about the dependence between the first and second one):
- one about the default keyboard choice for Vietnamese in gfxboot, currently VN but we need US (straight, not intl) ;
- one about the default input method when running GTK2, currently VIQR (automatically chosen because of the "vi" locale) but we need SCIM ;
- one about input method to add to language-support-vi, currently scim-tables-additional (which brings VIQR only) but we need instead scim-m17n AND m17n-db (which requires m17n-lib in turn) which brings VIQR (Vietnamese abroad), Telex (North Vietnam) and VNI (South Vietnam).

Note on the last point that Ubuntu (in fact Debian) currently only provides the 1.3.1 version of m17n-db (and m17n-lib), which is usable enough but does not allow typing "freely" (which means putting the accentuation at the end of the word). This "free typing" thing is provided only from m17n-db 1.4.0 which I did package into our HanoiLUG repository and in my PPA ( https://launchpad.net/~progfou/+archive ).

Hope this help. I hope these problems are not concerned by the current feature freeze (=> I hope we did not announce them too late...) since it's badly required here. The current situation is that we have to remaster the Ubuntu desktop CD at each release to try to enforce these policy a bit (but probably not doing this "The Right Way"™...).

Feel free to ask us more information or even to ask us to re-report these bugs independently, in which case help on choosing the right package assignments would be greatly appreciated.

Cheers, J.C.

Revision history for this message
Jean Christophe André (progfou) wrote :

Oh, and to answer Colin about console-setup (which is about console mode, isn't it?), in fact there are problems there too but far less important (IMHO) than in boot and graphic mode. This is because Vietnamese people are used to type without accents and so can deal with it if required.

But ideally, we would need at a minimum correct accentuated characters display in console mode, which should be easily provided by just choosing some Vietnamese font like Vietnamese-Fixed or viscii10 or any other containing all Vietnamese pre-composed characters. I did'nt test this part much though...

Requiring Unicode Level 2 for combining characters in console would be too much I guess, since the Linux kernel console code probably don't handle it, but we don't really need it anyway: pre-composed characters are enough to display any Vietnamese word correctly.

About Vietnamese typing in console mode, we'll probably have to wait for the next Ubuntu release because we do not have any good solution to propose right now. We know of some, like using kernel module or library preloading, but personally I really dislike these kind of solutions and I'm looking for something more like tty input processing interception in user space.

Cheers, J.C.

Revision history for this message
Colin Watson (cjwatson) wrote :

FYI the defaults in console-setup are used during installation to set the keyboard layout choices throughout the system, i.e. at the boot loader, at the console, and at X.

Revision history for this message
Jean Christophe André (progfou) wrote :

Ok. I've just check the console-setup package content and now I can see where it does apply. But yet I'm not sure to understand when the console-setup debconf options are configured during the installation. Is it from gfxboot or later from ubiquity?

In fact we face problem as soon as we choose the "Vietnamese" language in gfxboot since it does imply a "vn" keyboard which really should be "us" instead.

Then we face another problem when we run GTK2 applications (first with GDM, then with Ubiquity). It's because GTK2 has one of its default immodules (im-viqr.so) hinted for the "vi" language (see the result of "gtk-query-immodules-2.0") and so being automatically selected when in the "vi_VN" language environment. Here we probably need some GTK_IM_MODULE setting to disable this.

In a final Vietnamese configuration we set it to scim, and I think it may apply to most of, if not all, other languages too. So I would recommend to let it be the default, especially before running Ubiquity.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package console-setup - 1.21ubuntu3

---------------
console-setup (1.21ubuntu3) hardy; urgency=low

  * Set default layout for Vietnam to 'us' (LP: #191451).
  * Treat 'any' as a synonym for 'NoSymbol' in XKB input files (LP: #93077).

 -- Colin Watson <email address hidden> Tue, 26 Feb 2008 14:05:12 +0000

Changed in console-setup:
status: Incomplete → Fix Released
Revision history for this message
Colin Watson (cjwatson) wrote :

Jean: console-setup's choices are duplicated in gfxboot-theme-ubuntu (I've dealt with this), and then later handled again from ubiquity. This should all be in place now.

Revision history for this message
Colin Watson (cjwatson) wrote :

I'm not entirely sure what to do with the rest of this, but I'm passing it over to scim as the best holding area. Arne?

Revision history for this message
Ming Hua (minghua) wrote : [Bug 191451] Re: Bad default choices for Vietnamese installation

On Mon, Mar 03, 2008 at 03:21:05PM -0000, Colin Watson wrote:
>
> I'm not entirely sure what to do with the rest of this, but I'm passing
> it over to scim as the best holding area. Arne?

Just a comment in case Arne doesn't reply soon -- the viqr input method
is provided by scim-tables-additional (from scim-tables source package),
and I've heard that there are better Vietnamese support in the scim-m17n
package (needs a quite recent version of scim-m17n, though).

Ming
2008.03.04

Revision history for this message
Jean Christophe André (progfou) wrote :

About SCIM (and everything else), look at my first comment on February 26, 2008, detailing all requirements.

Revision history for this message
Ming Hua (minghua) wrote :

I see. Since the input method provided by GTK+ and the one provided by scim-tables are both named viqr, I was misunderstanding the situation.

Revision history for this message
Arne Goetje (arnegoetje) wrote :

I have added scim-m17n to the language-support-input-vi metapackage.
This should pull the packages in automatically when installing language-support for Vietnamese.
However, we cannot choose a specific IM to be the 'default' when the scim applet shows up. The user has to select the module he wants to use.
(Also, we would have the problem that two IMs (Telex and VNI) are used in VN, so which one to choose?)
If m17n 1.4.0 is required for Hardy, we would need a Freeze Exception...

As m17n in Debian unstable is still at 1.3.x, I'm wondering why it hasn't been updated yet. The latest available version 1.5.1.
I will ping the debian maintainer about it. Maybe there is a reason we don't know of why it hasn't been packaged yet.

Revision history for this message
Vu Do Quynh (vu-do-quynh) wrote :

Arne said:
(Also, we would have the problem that two IMs (Telex and VNI) are used in VN, so which one to choose?)

It is better to keep both, if possible, as Telex IM is common in the North of Vietnam and VNI IM is commoner is Southern Central and Southern Vietnam.
If only one of Telex or VNI can be kept, I would toss a coin to choose either ones ;-)

Revision history for this message
Jean Christophe André (progfou) wrote :

Thanks for this!

About which SCIM method to select, no problem. As soon as all required Vietnamese input methods are available, it's alright. I also think it's better to have no default for this for "political" reasons (classic North/South conflicts). ;-)

The m17n 1.4.0 is not absolutely required, but...
The 1.3.1 version has a nasty backspace bug: you have to use shift backspace, or comment two lines in the input method source file.
The "free typing" (entering tone mark at the end of the word) is highly appreciated and only (correctly) available from version 1.4.0.

About the m17n in Debian, there have been some wishlist bug reports for upgrade but without reaction (I didn't check recently though). I had no specific problem to package the 1.4.0. I just re-used the packaging information from Debian to do it, without much change (disabled patch for Tamil included upstream). Thanks for taking this.

I have finally downloaded Hardy Alpha 6 successfully and will now be able to test the Colin correction included.

Revision history for this message
Arne Goetje (arnegoetje) wrote :

The original debian maintainer of libm17n is MIA, so Harshula, who also packaged the -contrib, -docs and -db packages, took it over and is currently working on packaging the latest upstream version. So, it seems we still have to wait a week or two until we have up-to-date packages.

I hope this waiting time is acceptable, if not, we'll have to discuss what to do.

One option would be to fix the backspace issue in the Telex IM in 1.3.1 and use that one for Hardy.

Revision history for this message
Jean Christophe André (progfou) wrote :

Wait is always acceptable if we get what we want right in time! ;-)
Do you think he could do it soon enough to let it in Hardy?
If not, I've attached what is needed to correct the backspace issue.

Revision history for this message
Jean Christophe André (progfou) wrote :

I've just tested Hardy Alpha6 and here are my comments about Vietnamese support:
- default keyboard is now US at boot, in Xorg and Ubiquity, thanks to Colin Watson
- default GTK input method is "system" which still activate VIQR by default because of the im-viqr.so im-module hint for "vi" environment

Here I can see two options:
- let SCIM be the default input method in GTK => it's definitely the preferred long term solution (for Vietnamese and probably others too) but it sometime requires tuning before being fully usable, especially when mixing different locales/keyboard (eg Vietnamese with AZERTY), so this solution would require more testing and would not be appropriate right before the Hardy release ;
- disable the hint for Vietnamese locale in the im-viqr.so GTK im-module so GTK input will just use straight Xorg keyboard which is now US by default => it's probably the preferred solution here since it would affect only Vietnamese support and nothing else, so even in the case it would create troubles they would be only ours. ;-)

Revision history for this message
Sebastien Bacher (seb128) wrote :

what gtk changes do you expect there? could you summarize the issue?

Revision history for this message
Jean Christophe André (progfou) wrote :

Answer to Sebastien Bacher:

Problem summary:
When choosing some Vietnamese locale like "vi_VN", GTK is activating its VIQR input method automatically (because of im-module hint) which leads to unexpected typing (for most Vietnamese people, not used to this input method) and even loss of characters in input fields (eg an 'e' at the end of a word will be loosed if you don't type a space after it). Because of this we often get complaints about typing being wrong (like password not working) which leads to bad Ubuntu experience for Vietnamese people.

What we expect now:
We would like to get the VIQR GTK input method module *not activated by default* for a Vietnamese locale. It is probably easy to solve by just removing the "vi" hint from the GTK "im-viqr.so" im-module. I think it's the last step to solve this "Bad default choices for Vietnamese" now.

Don't hesitate to ask more or contact me directly as needed. TIA, J.C.

Revision history for this message
Jean Christophe André (progfou) wrote :

[More Hardy Alpha6 Vietnamese support tests]

It seems it's not exactly what I supposed... I see a strange GTK behavior here!
When running a gnome-terminal, the default GTK input method is "system" and VIQR is *not* activated.
When running ubiquity, the default GTK input method is also "system" but the VIQR input method *is* activated.
You can see the difference between input results in the attached screenshot.
So here I'm puzzled... I have no clue why the behavior is different between ubiquity or gnome-terminal, both being GTK compliant applications AFAIK...

The good news is that GTK doesn't seem to "eat" characters anymore: when I type a final 'e' and then press Enter I still get the 'e' in the input. But still the VIQR input method being running by default is an unwanted feature. Let's give another example: people typing a password containing "e^" will get it entered as "e^" in console (or any other "not-through-GTK" input) but as "ê" in GTK, leading to password problems confusion.

Revision history for this message
Ming Hua (minghua) wrote :

Hi Sebastien, hi Jean Christophe,

My understanding (which may well be wrong) is that Jean Christophe doesn't like the VIQR input method provided by GTK+ to be enabled by default for Vietnamese locales. The input method is provided by file /usr/lib/gtk-2.0/2.10.0/immodules/im-viqr.so, and its default status can be seen from the following two lines in /usr/lib/gtk-2.0/2.10.0/immodule-files.d/libgtk2.0-0.immodules:

"/usr/lib/gtk-2.0/2.10.0/immodules/im-viqr.so"
"viqr" "Vietnamese (VIQR)" "gtk20" "/usr/share/locale" "vi"

Revision history for this message
Ming Hua (minghua) wrote :

Attached is my COMPLETELY UNTESTED patch to show what I think changes need to be done for GTK+. Please don't apply it unless some Vietnamese user has confirmed that it does what he/she desires. I also don't quite understand how input method choosing works after the backporting of the change in GTK+ 2.16 that makes the right-click menu shows "System" instead of what $GTK_IM_MODULE specifies, so this patch may be on the wrong path.

Anyway, hope this at least helps the communication between the Vietnamese users and the GTK+ maintainers.

Revision history for this message
Jean Christophe André (progfou) wrote :

Thanks Ming Hua for you summary which is absolutely exact.
Your patch seems quite appropriate too, at least for the previous known GTK+ behavior.
Can somebody generate a .deb from this? Else I'll try through my PPA, but I'm not sure to have time for this today...

Revision history for this message
Arne Goetje (arnegoetje) wrote :

I applied the patch to m17n-db 1.3.1 to fix the Backspace problem in Telex IM.

Source package is on people.ubuntu.com/~arne/m17n-db/

Please sponsor.

(I'm afraid we won't be able to get m17n 1.5.1 into hardy, we are out of time. Beta freeze is in effect now. Unless of course someone steps up to do intensive testing for 1.5.1 to make sure there is no regression and applies for a Feature Freeze exception...)

Revision history for this message
Jean Christophe André (progfou) wrote :

We could organize intensive testing for Vietnamese, but I'm afraid we won't be able to test every single language provided by m17n-db... :-/

About the language-support-input-vi metapackage, I do confirm that adding scim-m17n is enough and does work from Hardy Alpha 6 (I've just tested it). It was not enough up to and including Gutsy because libm17n-0 didn't depend on m17n-db which is the case now.

Revision history for this message
Jean Christophe André (progfou) wrote :

not really a scim problem, and a fixed has been done for m17n-db to get a minimal working environment

Changed in scim:
status: Confirmed → Fix Committed
Revision history for this message
Jean Christophe André (progfou) wrote :

(not sure about what is the exact meaning of "Fix Committed" in term of bug management...)

Changed in m17n-db:
status: Fix Committed → In Progress
Changed in gtk+2.0:
assignee: nobody → desktop-bugs
importance: Undecided → Medium
milestone: none → ubuntu-8.04
status: New → Confirmed
Changed in gtk+2.0:
status: Confirmed → Incomplete
Revision history for this message
Steve Langasek (vorlon) wrote :

marking as 'confirmed' for gtk+2.0, I don't see any further information missing here; I think the request to not make VIQR the default input method for the vi_VN locale is clear.

Changed in gtk+2.0:
status: Incomplete → Confirmed
Revision history for this message
Sebastien Bacher (seb128) wrote :

did anybody try the gtk change?

Revision history for this message
Sebastien Bacher (seb128) wrote :

the change seems to be alright, I've some question though, would it be suitable for upstream or debian or do you consider it only as an hardy workaround? and could somebody try the change and confirm that's doing what is expected?

Revision history for this message
Vu Do Quynh (vu-do-quynh) wrote :

Hi,

I have downloaded Ubuntu 8.04 beta and did a test installation from the live CD Desktop i386.

F2 select Vietnamese (Tiêng Viêt)
F3 US English keyboard is selected OK

Start installation, selection of Vietnamese as language for install ; English American keyboard is selected OK

As far as I've seen, all the problems reported with the declaration of this bug seem to have gone.

Only thing remaining, when typing a vowel "a, e, i, o, u, y" it is still underlined while waiting for the next keyboard strike, however a strike sequence like "e." results effectively in "e.", not in "ẹ".

There seems also to be new issues for typing Vietnamese in Hardy Heron (beta), as the procedure followed to enable Vietnamses typing with SCIM, as described in http://www.hanoilug.org/dokuwiki/soft:scimtelex for Feisty and Gutsy, seems not to work any more with Hardy Heron (beta)

Revision history for this message
Jean Christophe André (progfou) wrote :

Answer to Sebstien Bacher:
  Sorry for current delay in my answers: I'm currently busy working longer than planed in Hồ Chí Minh City and have no time to do the requested test... Could somebody else test it please?
  But I'm not sure every Vietnamese people subscribed to this bug and interested in testing are able to rebuild a package including the patch proposed. Could somebody build such a package for them please?

Answer to Vũ Đỗ Quỳnh:
  Please check bug 103328: you should not need to follow our procedure anymore since it has been integrated already. But it has been done one day after the beta release so the correction is probably not available in the beta.

More on all this when I come back next week, sorry...

Revision history for this message
Jean Christophe André (progfou) wrote :

Forgot to answer on one point to Sebastien Bacher:
  This change is about disabling Vietnamese typing by default in GTK+, independently of the distribution, so is suitable for upstream, IMHO.
  But, we ask for this to be included in Hardy to help us promote Ubuntu deployment using a LTS with already good Vietnamese support and directly from the original Ubuntu, not the derivative some people started to do for Vietnam (myself, but others too).

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package gtk+2.0 - 2.12.9-2ubuntu2

---------------
gtk+2.0 (2.12.9-2ubuntu2) hardy; urgency=low

  * debian/patches/022_disable-viqr-im-for-vi-locale.patch:
    - change from Ming Hua, don't activate the viqr input method for Vietnamese
      (lp: #191451)

 -- Sebastien Bacher <email address hidden> Sat, 29 Mar 2008 20:30:22 +0100

Changed in gtk+2.0:
status: Confirmed → Fix Released
Revision history for this message
Sebastien Bacher (seb128) wrote :

The change seems to be correct, I've uploaded it to hardy so testing will be easier

Revision history for this message
Daniel Holbach (dholbach) wrote :

m17n-db patch was uploaded.

Changed in m17n-db:
status: In Progress → Fix Released
Revision history for this message
Jean Christophe André (progfou) wrote :

I have remastered the Hardy beta LiveCD after having updated the gtk+2.0 package and I do confirm that VIQR is not anymore activated by default for Vietnamese locale!

I have also installed the scim-m17n package to test m17n-db bugs corrections and I do confirm that Telex is now working appropriately (no more backspace or "aff" bugs) !

So this bug can now be closed. Many thanks!

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.