When to support Unicode Tibetan fonts?

gpontop · Post by **gpontop** » Fri Mar 20, 2009 1:42 am

FWJ · Post by **FWJ** » Fri Mar 20, 2009 9:12 am

Soon, I hope. Maybe a few months? I wish I could give you a better answer.

I realize you only asked a simple question (a good one) but perhaps it will be useful if I explain further.

I am working on the conversion every day and have been for several months.
I have all the basics working including Unicode Copy/Paste and RTF Import.
I works pretty well on Win2K/XP/Vista, and the BO Tibetan (PRC) keyboard works on Vista.
However, there are several difficult issues to solve.

The problem is that I am trying to create a version that supports Unicode and also continues to provide support for ANSI fonts *exactly* like the current version. This is proving VERY difficult. Mostly because I want it to work correctly for any user without requiring some special system configuration.

The standard way to display Unicode on a Windows system is with Microsoft's Uniscribe DLL (usp10.dll). Uniscribe helps a program display Unicode fonts and is very important for fonts that are considered "complex script". Unicode Tibetan is considered "complex script" and Uniscribe is able to do a good job of displaying it.

The problem is that once Uniscribe is enabled, the system code that displays older ANSI fonts starts to fail in strange ways. Even worse, the failures are dependent on what has been installed on the system. It depends on which version of the Uniscribe DLL is available on the system. As an example, about 20 characters in the TibetanMachine typeface are no longer displayed if you use certain Uniscribe versions; but do appear with other versions. Even if I find a Uniscribe version that mostly works I can't distribute it because Microsoft does not allow that.

The task for now is to try many ways to work around these issues.

evelynedechen · Post by **evelynedechen** » Sat Jun 20, 2009 12:01 pm

Thank you for working so hard on this.

Evelyne

FWJ · Post by **FWJ** » Mon Jun 22, 2009 4:29 pm

Thank you for the kind note Evelyne.

However, the news is not good. I have just returned from India/Nepal where I visited a half dozen Monasteries, the LTWA and TCRC and other Tibetan publishing organizations. I found that PechaMaker is appreciated and well received. However, I did not find a single Tibetan who was actually using Unicode for any project or document. Only about 3% of the computers available were running Vista. I also found that users were having to struggle to configure and maintain their computers; understandably there was just very little understanding of how things work.

Currently only Vista users with Service Pack 1 installed would be able to simply install PM and get good results for Tibetan Unicode. It is possible to setup earlier Vista and XP systems to work correctly but it takes several exact steps. I believe it is unacceptable to require these users to take those steps since deviation could adversely affect the machine, or fail to give good results. So, I am choosing to not distribute the Unicode version of PM (3.x) any time soon because it would only work correctly for updated Vista users and would just disappoint other users.

I'll keep a watch out for some acceptable solution but I am convinced that releasing now would just not be helpful for this user base.

jeffchwu · Post by **jeffchwu** » Mon Jun 22, 2009 6:30 pm

Hi, Fwj,

Is it possible to put usp10.dll on pechamaker's work path and make it work with pechamaker? Compare to put it on system32, it easy to teach other guys to make it work!

Best regards
Jeff

FWJ · Post by **FWJ** » Sun Jun 28, 2009 9:34 pm

Hi Jeff,
Yes, putting the correct usp10.dll file on PM's work path is all it takes to make it work.
That seems simple enough to say and to do; but mistakes can be made even with that.
My belief is that it is inappropriate to release without an installer that can take care of everything and assure good results for any user's system.

Some things I am thinking of:

The whole point of creating a pecha is to look at it.
It is unacceptable to provide a solution that does not provide good looking Tibetan results for all users.
Most people are not likely to read detailed instructions; and the target user base does not necessarily read English well.
At this point there is only one version of usp10.dll that I know of that actually works. Any other will give poor results.
If someone did try to jam any usp10.dll version into their system32\ then some ANSI Tibetan fonts could stop working; in very subtle ways.
And the problems with inexact installation are subtle.
For example, certain characters can just disappear, sometimes text looks fine on the screen but is blank when printing.
These things all depend on which OS and service pack you have installed.

I just think that the point is to be useful, and it is not useful to distribute an incomplete solution.
I haven't given up, I am still looking at it.

jeffchwu · Post by **jeffchwu** » Tue Jun 30, 2009 1:03 pm

Hi, Fwj

Thanks for your hard work first. ^_^

Sometimes, I get headache to change different font to different guy. Some people like this font, some body like that font.

And change the pecha to another font always make new problem.

So that's why I hope Unicode can be come up early. If every font become to unicode. That's easy to change to each other.

BTW, put USP10.DLL inside the install maybe not a good idea, I agree with you. It just one easy way to make it work.

I do not know programing well! Is it possibel select local USP10.DLL or system with option on PechaMaker?? If could, it is easy let somebody try and error. (include debuging and try new version USP10.DLL)

Sorry about to ask some many question with my poor English.

Jeff

FWJ · Post by **FWJ** » Wed Jul 01, 2009 8:13 pm

Hi Jeff,
No worries, your English works just fine.
I appreciate your questions because I think other people are wondering about these things too.

I completely agree that changing to Unicode will make things much easier; for users and for programmers. Unicode is fantastically useful for handling Tibetan text (converting, indexing, searching, spelling, etc). These text manipulation problems are solved; but the display problem is not solved.

Many Unicode capable editors are already available so it is easy to enter Tibetan Unicode. But PM is all about display. If the displayed text looks bad then it is useless; and Microsoft has made it impossible to insure proper display on Windows systems.

To answer your question, yes, a user could control which usp10.dll file PM will use, just by placing placing it in the PM install directory. But all versions look bad except for one, and I can't help make sure that all users get that version. It is not acceptable to make users go find a copy on some other machine, or to crawl the web and make a risky download.

As you mention, the ANSI font situation was a real headache, and Unicode is a huge leap forward in simplifying things. But releasing Unicode PM now will not help simplify things further for anyone other than the latest Vista users. I'm not willing to release "for Vista only".

jeffchwu · Post by **jeffchwu** » Thu Jul 02, 2009 12:25 pm

Hi, Fwj,

Sorry, I do not know that issue about "for Vista only".
For new version PechaMaker, it still can dispaly ANSI font, right?
If it work for both Ansi and Unicode, the user can select which one he like. So if user cannot display Unicode font well, he have two choice.
1) Keep use Ansi document
2) Type the unicode document and make pecha on old system/machine, but print at new system/machine. Due to that's just display problem, I think that's good idea.

Hoo, the only issue is maybe it is different page/line no. in different system, so we cannot find the wrong word to correct to right word. But we can search very easy, right?

Best regards
Jeff

FWJ · Post by **FWJ** » Tue Jul 07, 2009 9:21 am

Hi Jeff. Sorry for the delay, out of town again.

It seems that you are very comfortable with computers and you are able to do the extra things it would take to make Unicode work. You are pointing out the very steps it would take to make it work so I am sure you could do this. However this is not true for most users.

What I am trying to say is that I want to make PechaMaker work well for almost anyone. Users should not have to do more complex and unusual things to make it work correctly. What I mean by "Vista only" is that the only release I know how to make at this point would only work well on a Vista system.

Yes, 3.x does support both ANSI and Unicode. But it does not seem reasonable to offer a Unicode version when only the ANSI will work correctly for most users. If standard XP users try Unicode, many characters will look bad and they will not know why. How would they know they must switch to a Vista machine; and what if they do not have access to a Vista machine?

jeffchwu · Post by **jeffchwu** » Wed Jul 08, 2009 1:25 pm

Hi, Fwj,

Hooo, I know it diffcult to let others know the issue, and help them to fix it.
Sorry, I just like to get a work-system first, then wait it become profect and profect.
Thanks your information, I will wait 3.x version.

I will pray for it @^__^@

Jeff

cfynn · Post by **cfynn** » Fri Jul 17, 2009 8:36 am

A version of Uniscribe that works with Tibetan is already on a users system if:

1. user is using Windows Vista or Windows 7
2. user has MS Office 2003 SP 2 or above installed
3. user has Miicrosoft's free Word Viewer installed
4. user has manually updsated usp10.dll in Windows\System32

Could the PM installer look for a version of Uniscribe supporting Tibetan already on the users system and if available make a copy in the installation directory of PechaMaker (not necessary in case 1 & 4)

Office 2003 puts a copy of a private USP10.DLL in
\Program Files\Common Files\Microsoft Shared\OFFICE11\

Office 2007 puts a copy of a private USP10.DLL in
\Program Files\Microsoft Office\Office12

so those are places the installer could look.

If there is no version of Uniscribe which supports Tibetan already on system, suggest user download and install free Word Viewer from MS site then run PM install program again.

The most recent version of WordView is available for download from: <http://www.microsoft.com/downloads/deta ... laylang=en>

or: <http://tinyurl.com/MSWordView>

- Chris

jeffchwu · Post by **jeffchwu** » Fri Jul 17, 2009 9:24 am

Hi, Cfynn,

I think if possible, just use the DLL, do not copy. Due to that DLL will update by microsoft, it will not update to copyed-one.

B.R.
jeff

cfynn · Post by **cfynn** » Fri Jul 17, 2009 10:22 am

jeffchwu wrote:Hi, Cfynn,

I think if possible, just use the DLL, do not copy. Due to that DLL will update by microsoft, it will not update to copyed-one.

B.R.
jeff

Dear Jeff

If you install and updated USP10.DLL in a private program directory. such as the one PechaMaker is installed in. there is no problem -and a program starting from that directory will use that copy of the DLL rather than the one in the \Windows\System32\ directory. When looking for DLLs, Windows programs first look in the directory they start from and only then in the system directories and PATH.

This is why Tibetan Unicode will often work in Office 2003 but not in other applications under Windows XP - Office is using a private copy of USP10.DLL (that has Tibetan support) which is installed in the Office program directory rather than the system wide copy (which has no Tibetan support) installed in \Windows\System32\ .

It is much more difficult to update the USP10.DLL in the \Windows\System32\ directory since Windows security features will usually replace the original (old) copy of the file over the newer one you install.

- Chris

cfynn · Post by **cfynn** » Fri Jul 17, 2009 10:37 am

cfynn wrote:
jeffchwu wrote:Hi, Cfynn,

I think if possible, just use the DLL, do not copy. Due to that DLL will update by microsoft, it will not update to copyed-one.

B.R.
jeff
Dear Jeff

If you install and updated USP10.DLL in a private program directory. such as the one PechaMaker is installed in. there is no problem -and a program starting from that directory will use that copy of the DLL rather than the one in the \Windows\System32\ directory. When looking for DLLs, Windows programs first look in the directory they start from and only if not found there, then it looks in the system directories and PATH.
(This allows Windows to use different versions of the same DLL for different programs.)

This is the reason Tibetan Unicode will often work in Office 2003 but not in other applications under Windows XP - Office is using a private copy of USP10.DLL (that has Tibetan support) which is installed in the Office program directory rather than the system wide copy (which has no Tibetan support) installed in \Windows\System32\ .

It is much more difficult to update the USP10.DLL in the \Windows\System32\ directory since Windows security features will usually replace the original (old) copy of the file over the newer one you install.

Copying an updated USP10.DLL into a program directory is the method Microsoft themselves recommend in similar situations.

- Chris