From a9ef3e3cfd39661119c9658ad82b127f8b155494 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=86var=20Arnfj=C3=B6r=C3=B0=20Bjarmason?= Date: Sat, 10 Jul 2010 20:47:47 +0000 Subject: [PATCH 1/9] Add infrastructure for translating Git with gettext MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit All of the interface messages in Git core are currently hardcoded in English. Change that by optionally enabling translation of the core C, Shell and Perl programs via GNU or SunOS gettext. If you set the appropriate LC_* variables Git will speak your language, provided that someone has submitted a translation. If gettext isn't available, or if Git is compiled with NO_GETTEXT=YesPlease, then Git fall back on its previous behavior of only speaking English. When using ./configure the autoconf script will auto-detect if the gettext libraries are installed and act appropriately. With NO_GETTEXT=YesPlease gettext support will be #defined away for C programs. For Shell and Perl programs we rely on the git message catalog not being available. That's a reasonable assumption since then the message catalog won't be installed on the system during make install. The gettext wrappers that are provided in the patch are only the bare minimum required to begin translation work. In particular I haven't added wrappers for the gettext functions that enable plural support, or those that provide message context (msgctxt). Those can be added later. The intent is to start with a small subset and see what we need later, not to start with something that's unnecessarily large right away. Implementation and usage notes: * General: Gettext .mo files will be installed and looked for in the standard $(prefix)/share/locale path. GIT_TEXTDOMAINDIR can also be set to override that, but that's only intended to be used to test Git itself. * Perl: Perl code that wants to be localized should use the new Git::I18n module. It imports a __ function into the caller's package by default. Instead of using the high level Locale::TextDomain interface I've opted to use the low-level (equivalent to the C interface) Locale::Messages module, which Locale::TextDomain itself uses. Locale::TextDomain does a lot of redundant work we don't need, and some of it would potentially introduce bugs. It tries to set the $TEXTDOMAIN based on package of the caller, and has its own hardcoded paths where it'll search for messages. I found it easier just to completely avoid it rather than try to circumvent its behavior. In any case, this is an issue wholly internal Git::I18N. Its guts can be changed later if that's deemed necessary. See for a further elaboration on this topic. * Shell: Shell code that's to be localized should use the new git-sh-i18n library. It's just a wrapper for the system's gettext.sh. If gettext.sh isn't available we'll fall back gettext(1) if it's available. The latter is available without the former on Solaris, which has its own non-GNU gettext implementation. We also need to emulate eval_gettext() there. If neither are present we'll use a dumb printf(1) fall-through wrapper. I originally tried to detect if the system supported `echo -n' but I found this to be a waste of time. My benchmarks on Linux, Solaris and FreeBSD reveal that printf(1) is fast enough, especially since we aren't calling gettext() from within any tight loops, and unlikely to ever do so. This series has been tested by me on Ubuntu 10.04, Debian testing, FreeBSD 8.1 and SunOS 5.10, and by others on Mac OS X 10.6.3 (with Xcode 3.2.2) and openSUSE Factory (11.3, milestone 7). SunOS has its own non-GNU gettext implementation which this patch supports, although that may change in the future if it turns out that we need some GNU libintl features that SunOS doesn't provide. This patch is based on work by Jeff Epler who did the initial Makefile / C work, and a lot of comments from the Git mailing list, including Jonathan Nieder, Jakub Narebski, Johannes Sixt, Peter Krefting, Junio C Hamano, Thomas Rast and others. Signed-off-by: Ævar Arnfjörð Bjarmason Tested-By: John Wiegley Tested-by: Graham Anderson Signed-off-by: Junio C Hamano --- .gitignore | 2 + INSTALL | 12 ++++ Makefile | 69 ++++++++++++++++++++- config.mak.in | 2 + configure.ac | 12 ++++ daemon.c | 3 + fast-import.c | 3 + gettext.c | 22 +++++++ gettext.h | 18 ++++++ git-sh-i18n.sh | 71 ++++++++++++++++++++++ git.c | 3 + http-backend.c | 3 + http-fetch.c | 3 + http-push.c | 3 + imap-send.c | 3 + perl/Git/I18N.pm | 91 ++++++++++++++++++++++++++++ perl/Makefile | 3 +- perl/Makefile.PL | 14 ++++- po/.gitignore | 1 + po/is.po | 47 +++++++++++++++ shell.c | 3 + show-index.c | 3 + t/lib-gettext.sh | 42 +++++++++++++ t/t0200-gettext.sh | 113 +++++++++++++++++++++++++++++++++++ t/t0200/test.c | 13 ++++ t/t0200/test.perl | 14 +++++ t/t0200/test.sh | 14 +++++ t/t0201-gettext-fallbacks.sh | 49 +++++++++++++++ t/t0202-gettext-perl.sh | 27 +++++++++ t/t0202/test.pl | 107 +++++++++++++++++++++++++++++++++ t/test-lib.sh | 2 + upload-pack.c | 3 + 32 files changed, 771 insertions(+), 4 deletions(-) create mode 100644 gettext.c create mode 100644 gettext.h create mode 100644 git-sh-i18n.sh create mode 100644 perl/Git/I18N.pm create mode 100644 po/.gitignore create mode 100644 po/is.po create mode 100644 t/lib-gettext.sh create mode 100755 t/t0200-gettext.sh create mode 100644 t/t0200/test.c create mode 100644 t/t0200/test.perl create mode 100644 t/t0200/test.sh create mode 100755 t/t0201-gettext-fallbacks.sh create mode 100755 t/t0202-gettext-perl.sh create mode 100644 t/t0202/test.pl diff --git a/.gitignore b/.gitignore index 14e2b6bde9..6c2b1939f3 100644 --- a/.gitignore +++ b/.gitignore @@ -125,6 +125,7 @@ /git-rm /git-send-email /git-send-pack +/git-sh-i18n /git-sh-setup /git-shell /git-shortlog @@ -204,3 +205,4 @@ *.pdb /Debug/ /Release/ +/share/ diff --git a/INSTALL b/INSTALL index 61086ab120..9c9a5fa6be 100644 --- a/INSTALL +++ b/INSTALL @@ -93,6 +93,18 @@ Issues of note: history graphically, and in git-gui. If you don't want gitk or git-gui, you can use NO_TCLTK. + - A gettext library is used by default for localizing Git. The + primary target is GNU libintl, but the Solaris gettext + implementation also works. + + We need a gettext.h on the system for C code, gettext.sh (or + Solaris gettext(1)) for shell scripts, and libintl-perl for Perl + programs. + + Set NO_GETTEXT to disable localization support and make Git only + use English. Under autoconf the configure script will do this + automatically if it can't find libintl on the system. + - Some platform specific issues are dealt with Makefile rules, but depending on your specific installation, you may not have all the libraries/tools needed, or you may have diff --git a/Makefile b/Makefile index 9aca8a16d9..c97cbd8eba 100644 --- a/Makefile +++ b/Makefile @@ -34,6 +34,15 @@ all:: # Define NO_EXPAT if you do not have expat installed. git-http-push is # not built, and you cannot push using http:// and https:// transports. # +# Define NO_GETTEXT if you don't want to build with Git with gettext +# support. Building it requires GNU libintl or another gettext +# implementation, and additionally libintl-perl at runtime. +# +# Define NEEDS_LIBINTL if you haven't set NO_GETTEXT and your system +# needs to be explicitly linked to -lintl. It's defined automatically +# on platforms where we don't expect glibc (Linux, Hurd, +# GNU/kFreeBSD), which includes libintl. +# # Define EXPATDIR=/foo/bar if your expat header and library files are in # /foo/bar/include and /foo/bar/lib directories. # @@ -281,6 +290,7 @@ infodir = share/info gitexecdir = libexec/git-core sharedir = $(prefix)/share gitwebdir = $(sharedir)/gitweb +localedir = $(sharedir)/locale template_dir = share/git-core/templates htmldir = share/doc/git-doc ifeq ($(prefix),/usr) @@ -294,7 +304,7 @@ lib = lib # DESTDIR= pathsep = : -export prefix bindir sharedir sysconfdir gitwebdir +export prefix bindir sharedir sysconfdir gitwebdir localedir CC = gcc AR = ar @@ -308,6 +318,8 @@ TCL_PATH = tclsh TCLTK_PATH = wish PTHREAD_LIBS = -lpthread PTHREAD_CFLAGS = +XGETTEXT = xgettext +MSGFMT = msgfmt export TCL_PATH TCLTK_PATH @@ -369,6 +381,7 @@ SCRIPT_SH += git-web--browse.sh SCRIPT_LIB += git-mergetool--lib SCRIPT_LIB += git-parse-remote SCRIPT_LIB += git-sh-setup +SCRIPT_LIB += git-sh-i18n SCRIPT_PERL += git-add--interactive.perl SCRIPT_PERL += git-difftool.perl @@ -534,6 +547,7 @@ LIB_H += userdiff.h LIB_H += utf8.h LIB_H += xdiff-interface.h LIB_H += xdiff/xdiff.h +LIB_H += gettext.h LIB_OBJS += abspath.o LIB_OBJS += advice.o @@ -575,6 +589,9 @@ LIB_OBJS += entry.o LIB_OBJS += environment.o LIB_OBJS += exec_cmd.o LIB_OBJS += fsck.o +ifndef NO_GETTEXT +LIB_OBJS += gettext.o +endif LIB_OBJS += graph.o LIB_OBJS += grep.o LIB_OBJS += hash.o @@ -747,6 +764,14 @@ EXTLIBS = # Platform specific tweaks # +# Platform specific defaults. Where we'd only like some feature on the +# minority of systems, e.g. if linking to a library isn't needed +# because its features are included in the GNU C library. +ifndef NO_GETTEXT + # Systems that use GNU gettext and glibc are the exception + NEEDS_LIBINTL = YesPlease +endif + # We choose to avoid "if .. else if .. else .. endif endif" # because maintaining the nesting to match is a pain. If # we had "elif" things would have been much nicer... @@ -762,11 +787,13 @@ ifeq ($(uname_S),Linux) NO_STRLCPY = YesPlease NO_MKSTEMPS = YesPlease HAVE_PATHS_H = YesPlease + NEEDS_LIBINTL = endif ifeq ($(uname_S),GNU/kFreeBSD) NO_STRLCPY = YesPlease NO_MKSTEMPS = YesPlease HAVE_PATHS_H = YesPlease + NEEDS_LIBINTL = endif ifeq ($(uname_S),UnixWare) CC = cc @@ -958,6 +985,7 @@ ifeq ($(uname_S),GNU) NO_STRLCPY=YesPlease NO_MKSTEMPS = YesPlease HAVE_PATHS_H = YesPlease + NEEDS_LIBINTL = endif ifeq ($(uname_S),IRIX) NO_SETENV = YesPlease @@ -1456,6 +1484,14 @@ ifdef GIT_TEST_CMP_USE_COPIED_CONTEXT export GIT_TEST_CMP_USE_COPIED_CONTEXT endif +ifdef NO_GETTEXT + COMPAT_CFLAGS += -DNO_GETTEXT +endif + +ifdef NEEDS_LIBINTL + EXTLIBS += -lintl +endif + ifeq ($(TCLTK_PATH),) NO_TCLTK=NoThanks endif @@ -1485,6 +1521,7 @@ ifndef V QUIET_BUILT_IN = @echo ' ' BUILTIN $@; QUIET_GEN = @echo ' ' GEN $@; QUIET_LNCP = @echo ' ' LN/CP $@; + QUIET_MSGFMT = @echo ' ' MSGFMT $@; QUIET_SUBDIR0 = +@subdir= QUIET_SUBDIR1 = ;$(NO_SUBDIR) echo ' ' SUBDIR $$subdir; \ $(MAKE) $(PRINT_DIR) -C $$subdir @@ -1513,7 +1550,9 @@ template_dir_SQ = $(subst ','\'',$(template_dir)) htmldir_SQ = $(subst ','\'',$(htmldir)) prefix_SQ = $(subst ','\'',$(prefix)) gitwebdir_SQ = $(subst ','\'',$(gitwebdir)) +sharedir_SQ = $(subst ','\'',$(sharedir)) +LOCALEDIR_SQ = $(subst ','\'',$(localedir)) SHELL_PATH_SQ = $(subst ','\'',$(SHELL_PATH)) PERL_PATH_SQ = $(subst ','\'',$(PERL_PATH)) PYTHON_PATH_SQ = $(subst ','\'',$(PYTHON_PATH)) @@ -1563,7 +1602,7 @@ ifndef NO_TCLTK $(QUIET_SUBDIR0)gitk-git $(QUIET_SUBDIR1) all endif ifndef NO_PERL - $(QUIET_SUBDIR0)perl $(QUIET_SUBDIR1) PERL_PATH='$(PERL_PATH_SQ)' prefix='$(prefix_SQ)' all + $(QUIET_SUBDIR0)perl $(QUIET_SUBDIR1) PERL_PATH='$(PERL_PATH_SQ)' prefix='$(prefix_SQ)' localedir='$(localedir_SQ)' all endif ifndef NO_PYTHON $(QUIET_SUBDIR0)git_remote_helpers $(QUIET_SUBDIR1) PYTHON_PATH='$(PYTHON_PATH_SQ)' prefix='$(prefix_SQ)' all @@ -1609,6 +1648,7 @@ sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \ -e 's|@SHELL_PATH@|$(SHELL_PATH_SQ)|' \ -e 's|@@DIFF@@|$(DIFF_SQ)|' \ -e 's/@@GIT_VERSION@@/$(GIT_VERSION)/g' \ + -e 's|@@LOCALEDIR@@|$(LOCALEDIR_SQ)|g' \ -e 's/@@NO_CURL@@/$(NO_CURL)/g' \ -e $(BROKEN_PATH_FIX) \ $@.sh >$@+ @@ -1936,6 +1976,21 @@ cscope: $(RM) cscope* $(FIND) . -name '*.[hcS]' -print | xargs cscope -b +pot: + $(XGETTEXT) --add-comments --keyword=_ --keyword=N_ --output=po/git.pot --language=C $(C_OBJ:o=c) t/t0200/test.c + $(XGETTEXT) --add-comments --join-existing --output=po/git.pot --language=Shell $(SCRIPT_SH) t/t0200/test.sh + $(XGETTEXT) --add-comments --join-existing --keyword=__ --output=po/git.pot --language=Perl $(SCRIPT_PERL) t/t0200/test.perl + +POFILES := $(wildcard po/*.po) +MOFILES := $(patsubst po/%.po,share/locale/%/LC_MESSAGES/git.mo,$(POFILES)) +MODIRS := $(patsubst po/%.po,share/locale/%/LC_MESSAGES/,$(POFILES)) +ifndef NO_GETTEXT +all:: $(MOFILES) +endif +share/locale/%/LC_MESSAGES/git.mo: po/%.po + @mkdir -p $(dir $@) + $(QUIET_MSGFMT)$(MSGFMT) -o $@ $< + ### Detect prefix changes TRACK_CFLAGS = $(subst ','\'',$(ALL_CFLAGS)):\ $(bindir_SQ):$(gitexecdir_SQ):$(template_dir_SQ):$(prefix_SQ) @@ -1965,6 +2020,7 @@ endif ifdef GIT_TEST_CMP_USE_COPIED_CONTEXT @echo GIT_TEST_CMP_USE_COPIED_CONTEXT=YesPlease >>$@ endif + @echo NO_GETTEXT=\''$(subst ','\'',$(subst ','\'',$(NO_GETTEXT)))'\' >>$@ ### Detect Tck/Tk interpreter path changes ifndef NO_TCLTK @@ -2056,6 +2112,11 @@ install: all $(INSTALL) $(ALL_PROGRAMS) '$(DESTDIR_SQ)$(gitexec_instdir_SQ)' $(INSTALL) -m 644 $(SCRIPT_LIB) '$(DESTDIR_SQ)$(gitexec_instdir_SQ)' $(INSTALL) $(install_bindir_programs) '$(DESTDIR_SQ)$(bindir_SQ)' +ifndef NO_GETTEXT + $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(sharedir_SQ)/locale' + (cd share && tar cf - locale) | \ + (cd '$(DESTDIR_SQ)$(sharedir_SQ)' && umask 022 && tar xof -) +endif $(MAKE) -C templates DESTDIR='$(DESTDIR_SQ)' install ifndef NO_PERL $(MAKE) -C perl prefix='$(prefix_SQ)' DESTDIR='$(DESTDIR_SQ)' install @@ -2204,6 +2265,10 @@ ifndef NO_TCLTK $(MAKE) -C git-gui clean endif $(RM) GIT-VERSION-FILE GIT-CFLAGS GIT-GUI-VARS GIT-BUILD-OPTIONS +ifndef NO_GETTEXT + $(RM) po/git.pot + $(RM) -r share/ +endif .PHONY: all install clean strip .PHONY: shell_compatibility_test please_set_SHELL_PATH_to_a_more_modern_shell diff --git a/config.mak.in b/config.mak.in index b4e65c32b2..011b103d1b 100644 --- a/config.mak.in +++ b/config.mak.in @@ -34,9 +34,11 @@ NO_CURL=@NO_CURL@ NO_EXPAT=@NO_EXPAT@ NO_LIBGEN_H=@NO_LIBGEN_H@ HAVE_PATHS_H=@HAVE_PATHS_H@ +NO_GETTEXT=@NO_GETTEXT@ NEEDS_LIBICONV=@NEEDS_LIBICONV@ NEEDS_SOCKET=@NEEDS_SOCKET@ NEEDS_RESOLV=@NEEDS_RESOLV@ +NEEDS_LIBINTL=@NEEDS_LIBINTL@ NEEDS_LIBGEN=@NEEDS_LIBGEN@ NO_SYS_SELECT_H=@NO_SYS_SELECT_H@ NO_D_INO_IN_DIRENT=@NO_D_INO_IN_DIRENT@ diff --git a/configure.ac b/configure.ac index 5601e8bac9..202e82084e 100644 --- a/configure.ac +++ b/configure.ac @@ -600,6 +600,12 @@ AC_CHECK_LIB([c], [basename], AC_SUBST(NEEDS_LIBGEN) test -n "$NEEDS_LIBGEN" && LIBS="$LIBS -lgen" +AC_CHECK_LIB([c], [gettext], +[NEEDS_LIBINTL=], +[NEEDS_LIBINTL=YesPlease]) +AC_SUBST(NEEDS_LIBINTL) +test -n "$NEEDS_LIBINTL" && LIBS="$LIBS -lintl" + ## Checks for header files. AC_MSG_NOTICE([CHECKS for header files]) # @@ -777,6 +783,12 @@ AC_CHECK_HEADER([paths.h], [HAVE_PATHS_H=]) AC_SUBST(HAVE_PATHS_H) # +# Define NO_GETTEXT if you don't have libintl.h +AC_CHECK_HEADER([libintl.h], +[NO_GETTEXT=], +[NO_GETTEXT=YesPlease]) +AC_SUBST(NO_GETTEXT) +# # Define NO_STRCASESTR if you don't have strcasestr. GIT_CHECK_FUNC(strcasestr, [NO_STRCASESTR=], diff --git a/daemon.c b/daemon.c index e22a2b7fa5..784f8975d4 100644 --- a/daemon.c +++ b/daemon.c @@ -3,6 +3,7 @@ #include "exec_cmd.h" #include "run-command.h" #include "strbuf.h" +#include "gettext.h" #include @@ -975,6 +976,8 @@ int main(int argc, char **argv) gid_t gid = 0; int i; + git_setup_gettext(); + git_extract_argv0_path(argv[0]); for (i = 1; i < argc; i++) { diff --git a/fast-import.c b/fast-import.c index 1e5d66ed0a..6497b0cfc0 100644 --- a/fast-import.c +++ b/fast-import.c @@ -156,6 +156,7 @@ Format of STDIN stream: #include "csum-file.h" #include "quote.h" #include "exec_cmd.h" +#include "gettext.h" #define PACK_ID_BITS 16 #define MAX_PACK_ID ((1< +#include +#include + +extern void git_setup_gettext(void) { + char *podir; + char *envdir = getenv("GIT_TEXTDOMAINDIR"); + + if (envdir) { + (void)bindtextdomain("git", envdir); + } else { + podir = (char *)system_path("share/locale"); + if (!podir) return; + (void)bindtextdomain("git", podir); + free(podir); + } + + (void)setlocale(LC_MESSAGES, ""); + (void)setlocale(LC_CTYPE, ""); + (void)textdomain("git"); +} diff --git a/gettext.h b/gettext.h new file mode 100644 index 0000000000..e02939ae36 --- /dev/null +++ b/gettext.h @@ -0,0 +1,18 @@ +#ifndef GETTEXT_H +#define GETTEXT_H + +#ifdef NO_GETTEXT +static inline void git_setup_gettext(void) {} +#else +extern void git_setup_gettext(void); +#endif + +#define N_(s) (s) +#ifdef NO_GETTEXT +#define _(s) (s) +#else +#include +#define _(s) gettext(s) +#endif + +#endif diff --git a/git-sh-i18n.sh b/git-sh-i18n.sh new file mode 100644 index 0000000000..698a000fe5 --- /dev/null +++ b/git-sh-i18n.sh @@ -0,0 +1,71 @@ +#!/bin/sh +# +# Copyright (c) 2010 Ævar Arnfjörð Bjarmason +# +# This is Git's interface to gettext.sh. Use it right after +# git-sh-setup as: +# +# . git-sh-setup +# . git-sh-i18n +# +# # For constant interface messages: +# gettext "A message for the user"; echo +# +# # To interpolate variables: +# details="oh noes" +# eval_gettext "An error occured: \$details"; echo +# +# See "info '(gettext)sh'" for the full manual. + +# Export the TEXTDOMAIN* data that we need for Git +TEXTDOMAIN=git +export TEXTDOMAIN +if [ -z "$GIT_TEXTDOMAINDIR" ] +then + TEXTDOMAINDIR="@@LOCALEDIR@@" +else + TEXTDOMAINDIR="$GIT_TEXTDOMAINDIR" +fi +export TEXTDOMAINDIR + +if test -z "$GIT_INTERNAL_GETTEXT_TEST_FALLBACKS" && type gettext.sh >/dev/null 2>&1 +then + # This is GNU libintl's gettext.sh, we don't need to do anything + # else than setting up the environment and loading gettext.sh + GIT_INTERNAL_GETTEXT_SH_SCHEME=gnu + export GIT_INTERNAL_GETTEXT_SH_SCHEME + + # Try to use libintl's gettext.sh, or fall back to English if we + # can't. + . gettext.sh +elif test -z "$GIT_INTERNAL_GETTEXT_TEST_FALLBACKS" && test "$(gettext -h 2>&1)" = "-h" +then + # We don't have gettext.sh, but there's a gettext binary in our + # path. This is probably Solaris or something like it which has a + # gettext implementation that isn't GNU libintl. + GIT_INTERNAL_GETTEXT_SH_SCHEME=solaris + export GIT_INTERNAL_GETTEXT_SH_SCHEME + + # Solaris has a gettext(1) but no eval_gettext(1) + eval_gettext () { + gettext_out=$(gettext "$1") + gettext_eval="printf '%s' \"$gettext_out\"" + printf "%s" "`eval \"$gettext_eval\"`" + } +else + # Since gettext.sh isn't available we'll have to define our own + # dummy pass-through functions. + + # Tell our tests that we don't have the real gettext.sh + GIT_INTERNAL_GETTEXT_SH_SCHEME=fallthrough + export GIT_INTERNAL_GETTEXT_SH_SCHEME + + gettext () { + printf "%s" "$1" + } + + eval_gettext () { + gettext_eval="printf '%s' \"$1\"" + printf "%s" "`eval \"$gettext_eval\"`" + } +fi diff --git a/git.c b/git.c index 265fa09d8d..95df6a6227 100644 --- a/git.c +++ b/git.c @@ -3,6 +3,7 @@ #include "cache.h" #include "quote.h" #include "run-command.h" +#include "gettext.h" const char git_usage_string[] = "git [--version] [--exec-path[=GIT_EXEC_PATH]] [--html-path]\n" @@ -490,6 +491,8 @@ int main(int argc, const char **argv) if (!cmd) cmd = "git-help"; + git_setup_gettext(); + /* * "git-xxxx" is the same as "git xxxx", but we obviously: * diff --git a/http-backend.c b/http-backend.c index 14c90c2e84..502103d50d 100644 --- a/http-backend.c +++ b/http-backend.c @@ -7,6 +7,7 @@ #include "run-command.h" #include "string-list.h" #include "url.h" +#include "gettext.h" static const char content_type[] = "Content-Type"; static const char content_length[] = "Content-Length"; @@ -550,6 +551,8 @@ int main(int argc, char **argv) char *cmd_arg = NULL; int i; + git_setup_gettext(); + git_extract_argv0_path(argv[0]); set_die_routine(die_webcgi); diff --git a/http-fetch.c b/http-fetch.c index 762c750d7a..b889c36994 100644 --- a/http-fetch.c +++ b/http-fetch.c @@ -2,6 +2,7 @@ #include "exec_cmd.h" #include "http.h" #include "walker.h" +#include "gettext.h" static const char http_fetch_usage[] = "git http-fetch " "[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url"; @@ -24,6 +25,8 @@ int main(int argc, const char **argv) int get_verbosely = 0; int get_recover = 0; + git_setup_gettext(); + git_extract_argv0_path(argv[0]); while (arg < argc && argv[arg][0] == '-') { diff --git a/http-push.c b/http-push.c index c9bcd11697..fc2c5f7f65 100644 --- a/http-push.c +++ b/http-push.c @@ -10,6 +10,7 @@ #include "remote.h" #include "list-objects.h" #include "sigchain.h" +#include "gettext.h" #include @@ -1791,6 +1792,8 @@ int main(int argc, char **argv) struct remote *remote; char *rewritten_url = NULL; + git_setup_gettext(); + git_extract_argv0_path(argv[0]); repo = xcalloc(sizeof(*repo), 1); diff --git a/imap-send.c b/imap-send.c index 1a577a0a09..c8fb1f9d42 100644 --- a/imap-send.c +++ b/imap-send.c @@ -25,6 +25,7 @@ #include "cache.h" #include "exec_cmd.h" #include "run-command.h" +#include "gettext.h" #ifdef NO_OPENSSL typedef void *SSL; #else @@ -1535,6 +1536,8 @@ int main(int argc, char **argv) git_extract_argv0_path(argv[0]); + git_setup_gettext(); + if (argc != 1) usage(imap_send_usage); diff --git a/perl/Git/I18N.pm b/perl/Git/I18N.pm new file mode 100644 index 0000000000..5918d68847 --- /dev/null +++ b/perl/Git/I18N.pm @@ -0,0 +1,91 @@ +package Git::I18N; +use 5.006002; +use strict; +use warnings; +use Exporter; +use base 'Exporter'; + +our $VERSION = '0.01'; + +our @EXPORT = qw(__); +our @EXPORT_OK = @EXPORT; + +sub __bootstrap_locale_messages { + our $TEXTDOMAIN = 'git'; + our $TEXTDOMAINDIR = $ENV{GIT_TEXTDOMAINDIR} || '++LOCALEDIR++'; + + require POSIX; + POSIX->import(qw(setlocale)); + # Non-core prerequisite module + require Locale::Messages; + Locale::Messages->import(qw(:locale_h :libintl_h)); + + setlocale(LC_MESSAGES(), ''); + setlocale(LC_CTYPE(), ''); + textdomain($TEXTDOMAIN); + bindtextdomain($TEXTDOMAIN => $TEXTDOMAINDIR); + + return; +} + +BEGIN +{ + # Used by our test script to see if it should test fallbacks or + # not. + our $__HAS_LIBRARY = 1; + + local $@; + eval { __bootstrap_locale_messages() }; + if ($@) { + # Tell test.pl that we couldn't load the gettext library. + $Git::I18N::__HAS_LIBRARY = 0; + + # Just a fall-through no-op + *__ = sub ($) { $_[0] }; + } else { + *__ = \&Locale::Messages::gettext; + } +} + +1; + +__END__ + +=head1 NAME + +Git::I18N - Perl interface to Git's Gettext localizations + +=head1 SYNOPSIS + + use Git::I18N; + + print __("Welcome to Git!\n"); + + printf __("The following error occured: %s\n"), $error; + +=head1 DESCRIPTION + +Git's internal Perl interface to gettext via L. If +L can't be loaded (it's not a core module) we +provide stub passthrough fallbacks. + +This is a distilled interface to gettext, see C +for the full interface. This module implements only a small part of +it. + +=head1 FUNCTIONS + +=head2 __($) + +L's gettext function if all goes well, otherwise our +passthrough fallback function. + +=head1 AUTHOR + +Evar ArnfjErE Bjarmason + +=head1 COPYRIGHT + +Copyright 2010 Evar ArnfjErE Bjarmason + +=cut diff --git a/perl/Makefile b/perl/Makefile index 4ab21d61b8..4e624fff90 100644 --- a/perl/Makefile +++ b/perl/Makefile @@ -5,6 +5,7 @@ makfile:=perl.mak PERL_PATH_SQ = $(subst ','\'',$(PERL_PATH)) prefix_SQ = $(subst ','\'',$(prefix)) +localedir_SQ = $(subst ','\'',$(localedir)) ifndef V QUIET = @ @@ -38,7 +39,7 @@ $(makfile): ../GIT-CFLAGS Makefile echo ' echo $(instdir_SQ)' >> $@ else $(makfile): Makefile.PL ../GIT-CFLAGS - $(PERL_PATH) $< PREFIX='$(prefix_SQ)' + $(PERL_PATH) $< PREFIX='$(prefix_SQ)' --localedir='$(localedir_SQ)' endif # this is just added comfort for calling make directly in perl dir diff --git a/perl/Makefile.PL b/perl/Makefile.PL index 0b9deca2cc..456d45bf40 100644 --- a/perl/Makefile.PL +++ b/perl/Makefile.PL @@ -1,4 +1,12 @@ +use strict; +use warnings; use ExtUtils::MakeMaker; +use Getopt::Long; + +# Sanity: die at first unknown option +Getopt::Long::Configure qw/ pass_through /; + +GetOptions("localedir=s" => \my $localedir); sub MY::postamble { return <<'MAKE_FRAG'; @@ -16,7 +24,10 @@ endif MAKE_FRAG } -my %pm = ('Git.pm' => '$(INST_LIBDIR)/Git.pm'); +my %pm = ( + 'Git.pm' => '$(INST_LIBDIR)/Git.pm', + 'Git/I18N.pm' => '$(INST_LIBDIR)/Git/I18N.pm', +); # We come with our own bundled Error.pm. It's not in the set of default # Perl modules so install it if it's not available on the system yet. @@ -33,6 +44,7 @@ WriteMakefile( NAME => 'Git', VERSION_FROM => 'Git.pm', PM => \%pm, + PM_FILTER => qq[\$(PERL) -pe "s<\\Q++LOCALEDIR++\\E><$localedir>"], MAKEFILE => 'perl.mak', INSTALLSITEMAN3DIR => '$(SITEPREFIX)/share/man/man3' ); diff --git a/po/.gitignore b/po/.gitignore new file mode 100644 index 0000000000..221000e1b0 --- /dev/null +++ b/po/.gitignore @@ -0,0 +1 @@ +/*.pot diff --git a/po/is.po b/po/is.po new file mode 100644 index 0000000000..95739f1faa --- /dev/null +++ b/po/is.po @@ -0,0 +1,47 @@ +msgid "" +msgstr "" +"Project-Id-Version: Git\n" +"PO-Revision-Date: 2010-06-05 19:06 +0000\n" +"Language-Team: Git Mailing List \n" +"Report-Msgid-Bugs-To: Git Mailing List \n" +"Last-Translator: Ævar Arnfjörð Bjarmason \n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" + +#: t/t0200/test.c:4 +msgid "See git help COMMAND for more information on a specific command." +msgstr "Sjá git help SKIPUN til að sjá hjálp fyrir tiltekna skipun." + +#. TRANSLATORS: This is a test. You don't need to translate it. +#: t/t0200/test.c:9 +msgid "TEST: A C test string" +msgstr "TILRAUN: C tilraunastrengur" + +#. TRANSLATORS: This is a test. You don't need to translate it. +#: t/t0200/test.c:12 +#, c-format +msgid "TEST: A C test string %s" +msgstr "TILRAUN: C tilraunastrengur %s" + +#. TRANSLATORS: This is a test. You don't need to translate it. +#: t/t0200/test.sh:8 +msgid "TEST: A Shell test string" +msgstr "TILRAUN: Skeljartilraunastrengur" + +#. TRANSLATORS: This is a test. You don't need to translate it. +#: t/t0200/test.sh:11 +#, sh-format +msgid "TEST: A Shell test $variable" +msgstr "TILRAUN: Skeljartilraunastrengur með breytunni $variable" + +#. TRANSLATORS: This is a test. You don't need to translate it. +#: t/t0200/test.perl:8 +msgid "TEST: A Perl test string" +msgstr "TILRAUN: Perl tilraunastrengur" + +#. TRANSLATORS: This is a test. You don't need to translate it. +#: t/t0200/test.perl:11 +#, perl-format +msgid "TEST: A Perl test variable %s" +msgstr "TILRAUN: Perl tilraunastrengur með breytunni %s" diff --git a/shell.c b/shell.c index e4864e04da..ba27c6be83 100644 --- a/shell.c +++ b/shell.c @@ -2,6 +2,7 @@ #include "quote.h" #include "exec_cmd.h" #include "strbuf.h" +#include "gettext.h" static int do_generic_cmd(const char *me, char *arg) { @@ -51,6 +52,8 @@ int main(int argc, char **argv) struct commands *cmd; int devnull_fd; + git_setup_gettext(); + /* * Always open file descriptors 0/1/2 to avoid clobbering files * in die(). It also avoids not messing up when the pipes are diff --git a/show-index.c b/show-index.c index 4c0ac138af..c2f5448536 100644 --- a/show-index.c +++ b/show-index.c @@ -1,5 +1,6 @@ #include "cache.h" #include "pack.h" +#include "gettext.h" static const char show_index_usage[] = "git show-index < "; @@ -11,6 +12,8 @@ int main(int argc, char **argv) unsigned int version; static unsigned int top_index[256]; + git_setup_gettext(); + if (argc != 1) usage(show_index_usage); if (fread(top_index, 2 * 4, 1, stdin) != 1) diff --git a/t/lib-gettext.sh b/t/lib-gettext.sh new file mode 100644 index 0000000000..831ee38e83 --- /dev/null +++ b/t/lib-gettext.sh @@ -0,0 +1,42 @@ +#!/bin/sh +# +# Copyright (c) 2010 Ævar Arnfjörð Bjarmason +# + +. ./test-lib.sh + +GIT_TEXTDOMAINDIR="$GIT_EXEC_PATH/share/locale" +GIT_PO_PATH="$GIT_EXEC_PATH/po" +export GIT_TEXTDOMAINDIR GIT_PO_PATH + +. "$GIT_EXEC_PATH"/git-sh-i18n + +if test_have_prereq GETTEXT +then + # is_IS.UTF-8 on Solaris and FreeBSD, is_IS.utf8 on Debian + is_IS_locale=$(locale -a | sed -n '/^is_IS\.[uU][tT][fF]-*8$/{ + p + q + }') + # Export it as an environmental variable so the t0202/test.pl Perl + # test can use it too + export is_IS_locale + + if test -n "$is_IS_locale" && + test $GIT_INTERNAL_GETTEXT_SH_SCHEME != "fallthrough" + then + # Some of the tests need the reference Icelandic locale + test_set_prereq GETTEXT_LOCALE + + # Exporting for t0202/test.pl + GETTEXT_LOCALE=1 + export GETTEXT_LOCALE + say "# lib-gettext: Found '$is_IS_locale' as a is_IS UTF-8 locale" + else + say "# lib-gettext: No is_IS UTF-8 locale available" + fi +else + # Only run some tests when we don't have gettext support + test_set_prereq NO_GETTEXT + say "# lib-gettext: No GETTEXT support available" +fi diff --git a/t/t0200-gettext.sh b/t/t0200-gettext.sh new file mode 100755 index 0000000000..522338d13f --- /dev/null +++ b/t/t0200-gettext.sh @@ -0,0 +1,113 @@ +#!/bin/sh +# +# Copyright (c) 2010 Ævar Arnfjörð Bjarmason +# + +test_description='Gettext support for Git' + +. ./lib-gettext.sh + +test_expect_success "sanity: \$GIT_INTERNAL_GETTEXT_SH_SCHEME is set (to $GIT_INTERNAL_GETTEXT_SH_SCHEME)" ' + test -n "$GIT_INTERNAL_GETTEXT_SH_SCHEME" +' + +test_expect_success 'sanity: $TEXTDOMAIN is git' ' + test $TEXTDOMAIN = "git" +' + +test_expect_success 'xgettext sanity: Perl _() strings are not extracted' ' + ! grep "A Perl string xgettext will not get" "$GIT_PO_PATH"/is.po +' + +test_expect_success 'xgettext sanity: Comment extraction with --add-comments' ' + grep "TRANSLATORS: This is a test" "$TEST_DIRECTORY"/t0200/* | wc -l >expect && + grep "TRANSLATORS: This is a test" "$GIT_PO_PATH"/is.po | wc -l >actual && + test_cmp expect actual +' + +test_expect_success 'xgettext sanity: Comment extraction with --add-comments stops at statements' ' + ! grep "This is a phony" "$GIT_PO_PATH"/is.po && + ! grep "the above comment" "$GIT_PO_PATH"/is.po +' + +test_expect_success GETTEXT 'sanity: $TEXTDOMAINDIR exists without NO_GETTEXT=YesPlease' ' + test -d "$TEXTDOMAINDIR" && + test "$TEXTDOMAINDIR" = "$GIT_TEXTDOMAINDIR" +' + +test_expect_success GETTEXT 'sanity: Icelandic locale was compiled' ' + test -f "$TEXTDOMAINDIR/is/LC_MESSAGES/git.mo" +' + +test_expect_success NO_GETTEXT "sanity: \$TEXTDOMAINDIR doesn't exists with NO_GETTEXT=YesPlease" ' + ! test -d "$TEXTDOMAINDIR" && + test "$TEXTDOMAINDIR" = "$GIT_TEXTDOMAINDIR" +' + +# TODO: When we have more locales, generalize this to test them +# all. Maybe we'll need a dir->locale map for that. +test_expect_success GETTEXT_LOCALE 'sanity: gettext("") metadata is OK' ' + # Return value may be non-zero + LANGUAGE=is LC_ALL="$is_IS_locale" gettext "" >zero-expect && + grep "Project-Id-Version: Git" zero-expect && + grep "Git Mailing List " zero-expect && + grep "Content-Type: text/plain; charset=UTF-8" zero-expect && + grep "Content-Transfer-Encoding: 8bit" zero-expect +' + +test_expect_success GETTEXT_LOCALE 'sanity: gettext(unknown) is passed through' ' + printf "This is not a translation string" >expect && + gettext "This is not a translation string" >actual && + eval_gettext "This is not a translation string" >actual && + test_cmp expect actual +' + +# xgettext from C +test_expect_success GETTEXT_LOCALE 'xgettext: C extraction of _() and N_() strings' ' + printf "TILRAUN: C tilraunastrengur" >expect && + printf "\n" >>expect && + printf "Sjá git help SKIPUN til að sjá hjálp fyrir tiltekna skipun." >>expect && + LANGUAGE=is LC_ALL="$is_IS_locale" gettext "TEST: A C test string" >actual && + printf "\n" >>actual && + LANGUAGE=is LC_ALL="$is_IS_locale" gettext "See git help COMMAND for more information on a specific command." >>actual && + test_cmp expect actual +' + +test_expect_success GETTEXT_LOCALE 'xgettext: C extraction with %s' ' + printf "TILRAUN: C tilraunastrengur %%s" >expect && + LANGUAGE=is LC_ALL="$is_IS_locale" gettext "TEST: A C test string %s" >actual && + test_cmp expect actual +' + +# xgettext from Shell +test_expect_success GETTEXT_LOCALE 'xgettext: Shell extraction' ' + printf "TILRAUN: Skeljartilraunastrengur" >expect && + LANGUAGE=is LC_ALL="$is_IS_locale" gettext "TEST: A Shell test string" >actual && + test_cmp expect actual +' + +test_expect_success GETTEXT_LOCALE 'xgettext: Shell extraction with $variable' ' + printf "TILRAUN: Skeljartilraunastrengur með breytunni a var i able" >x-expect && + LANGUAGE=is LC_ALL="$is_IS_locale" variable="a var i able" eval_gettext "TEST: A Shell test \$variable" >x-actual && + test_cmp x-expect x-actual +' + +# xgettext from Perl +test_expect_success GETTEXT_LOCALE 'xgettext: Perl extraction' ' + printf "TILRAUN: Perl tilraunastrengur" >expect && + LANGUAGE=is LC_ALL="$is_IS_locale" gettext "TEST: A Perl test string" >actual && + test_cmp expect actual +' + +test_expect_success GETTEXT_LOCALE 'xgettext: Perl extraction with %s' ' + printf "TILRAUN: Perl tilraunastrengur með breytunni %%s" >expect && + LANGUAGE=is LC_ALL="$is_IS_locale" gettext "TEST: A Perl test variable %s" >actual && + test_cmp expect actual +' + +test_expect_success GETTEXT_LOCALE 'sanity: Some gettext("") data for real locale' ' + LANGUAGE=is LC_ALL="$is_IS_locale" gettext "" >real-locale && + test -s real-locale +' + +test_done diff --git a/t/t0200/test.c b/t/t0200/test.c new file mode 100644 index 0000000000..93373b38f7 --- /dev/null +++ b/t/t0200/test.c @@ -0,0 +1,13 @@ +/* This is a phony C program that's only here to test xgettext message extraction */ + +const char help[] = + N_("See 'git help COMMAND' for more information on a specific command."); + +int main(void) +{ + /* TRANSLATORS: This is a test. You don't need to translate it. */ + puts(_("TEST: A C test string")); + + /* TRANSLATORS: This is a test. You don't need to translate it. */ + printf(_("TEST: A C test string %s"), "variable"); +} diff --git a/t/t0200/test.perl b/t/t0200/test.perl new file mode 100644 index 0000000000..36fba341ba --- /dev/null +++ b/t/t0200/test.perl @@ -0,0 +1,14 @@ +# This is a phony Perl program that's only here to test xgettext +# message extraction + +# so the above comment won't be folded into the next one by xgettext +1; + +# TRANSLATORS: This is a test. You don't need to translate it. +print __("TEST: A Perl test string"); + +# TRANSLATORS: This is a test. You don't need to translate it. +printf __("TEST: A Perl test variable %s"), "moo"; + +# TRANSLATORS: If you see this, Git has a bug +print _"TEST: A Perl string xgettext will not get"; diff --git a/t/t0200/test.sh b/t/t0200/test.sh new file mode 100644 index 0000000000..022d607f4c --- /dev/null +++ b/t/t0200/test.sh @@ -0,0 +1,14 @@ +# This is a phony Shell program that's only here to test xgettext +# message extraction + +# so the above comment won't be folded into the next one by xgettext +echo + +# TRANSLATORS: This is a test. You don't need to translate it. +gettext "TEST: A Shell test string" + +# TRANSLATORS: This is a test. You don't need to translate it. +eval_gettext "TEST: A Shell test \$variable" + +# TRANSLATORS: If you see this, Git has a bug +_("TEST: A Shell string xgettext won't get") diff --git a/t/t0201-gettext-fallbacks.sh b/t/t0201-gettext-fallbacks.sh new file mode 100755 index 0000000000..47ce4f6b6e --- /dev/null +++ b/t/t0201-gettext-fallbacks.sh @@ -0,0 +1,49 @@ +#!/bin/sh +# +# Copyright (c) 2010 Ævar Arnfjörð Bjarmason +# + +test_description='Gettext Shell fallbacks' + +GIT_INTERNAL_GETTEXT_TEST_FALLBACKS=YesPlease +export GIT_INTERNAL_GETTEXT_TEST_FALLBACKS + +. ./lib-gettext.sh + +test_expect_success "sanity: \$GIT_INTERNAL_GETTEXT_SH_SCHEME is set (to $GIT_INTERNAL_GETTEXT_SH_SCHEME)" ' + test -n "$GIT_INTERNAL_GETTEXT_SH_SCHEME" +' + +test_expect_success 'sanity: $GIT_INTERNAL_GETTEXT_TEST_FALLBACKS is set' ' + test -n "$GIT_INTERNAL_GETTEXT_TEST_FALLBACKS" +' + +test_expect_success 'sanity: $GIT_INTERNAL_GETTEXT_SH_SCHEME" is fallthrough' ' + test "$GIT_INTERNAL_GETTEXT_SH_SCHEME" = "fallthrough" +' + +test_expect_success 'gettext: our gettext() fallback has pass-through semantics' ' + printf "test" >expect && + gettext "test" >actual && + test_cmp expect actual && + printf "test more words" >expect && + gettext "test more words" >actual && + test_cmp expect actual +' + +test_expect_success 'eval_gettext: our eval_gettext() fallback has pass-through semantics' ' + printf "test" >expect && + eval_gettext "test" >actual && + test_cmp expect actual && + printf "test more words" >expect && + eval_gettext "test more words" >actual && + test_cmp expect actual +' + +test_expect_success 'eval_gettext: our eval_gettext() fallback can interpolate variables' ' + printf "test YesPlease" >expect && + eval_gettext "test \$GIT_INTERNAL_GETTEXT_TEST_FALLBACKS" >actual && + test_cmp expect actual +' + +test_done diff --git a/t/t0202-gettext-perl.sh b/t/t0202-gettext-perl.sh new file mode 100755 index 0000000000..428ebb0080 --- /dev/null +++ b/t/t0202-gettext-perl.sh @@ -0,0 +1,27 @@ +#!/bin/sh +# +# Copyright (c) 2010 Ævar Arnfjörð Bjarmason +# + +test_description='Perl gettext interface (Git::I18N)' + +. ./lib-gettext.sh + +if ! test_have_prereq PERL; then + skip_all='skipping perl interface tests, perl not available' + test_done +fi + +"$PERL_PATH" -MTest::More -e 0 2>/dev/null || { + skip_all="Perl Test::More unavailable, skipping test" + test_done +} + +# The external test will outputs its own plan +test_external_has_tap=1 + +test_external_without_stderr \ + 'Perl Git::I18N API' \ + "$PERL_PATH" "$TEST_DIRECTORY"/t0202/test.pl + +test_done diff --git a/t/t0202/test.pl b/t/t0202/test.pl new file mode 100644 index 0000000000..c2055fa8de --- /dev/null +++ b/t/t0202/test.pl @@ -0,0 +1,107 @@ +#!/usr/bin/perl +use 5.006002; +use lib (split(/:/, $ENV{GITPERLLIB})); +use warnings; +use strict; +use Test::More tests => 9; +use Git::I18N; +use POSIX qw(:locale_h); + +my $has_gettext_library = $Git::I18N::__HAS_LIBRARY; + +ok(1, "Testing Git::I18N version $Git::I18N::VERSION with " . + ($has_gettext_library + ? "Locale::Messages version $Locale::Messages::VERSION" + : "NO Perl gettext library")); +ok(1, "Git::I18N is located at $INC{'Git/I18N.pm'}"); + +ok($Git::I18N::VERSION, 'sanity: Git::I18N defines a $VERSION'); +{ + my $exports = @Git::I18N::EXPORT; + ok($exports, "sanity: Git::I18N has $exports export(s)"); +} +is_deeply(\@Git::I18N::EXPORT, \@Git::I18N::EXPORT_OK, "sanity: Git::I18N exports everything by default"); + +# prototypes +{ + # Add prototypes here when modifying the public interface to add + # more gettext wrapper functions. + my %prototypes = (qw( + __ $ + )); + while (my ($sub, $proto) = each %prototypes) { + is(prototype(\&{"Git::I18N::$sub"}), $proto, "sanity: $sub has a $proto prototype"); + } +} + +# Test basic passthrough in the C locale +{ + local $ENV{LANGUAGE} = 'C'; + local $ENV{LC_ALL} = 'C'; + local $ENV{LANG} = 'C'; + + my ($got, $expect) = (('TEST: A Perl test string') x 2); + + is(__($got), $expect, "Passing a string through __() in the C locale works"); +} + +# Test a basic message on different locales +SKIP: { + unless ($ENV{GETTEXT_LOCALE}) { + # Can't reliably test __() with a non-C locales because the + # required locales may not be installed on the system. + # + # We test for these anyway as part of the shell + # tests. Skipping these here will eliminate failures on odd + # platforms with incomplete locale data. + + skip "GETTEXT_LOCALE must be set by lib-gettext.sh for exhaustive Git::I18N tests", 2; + } + + # The is_IS UTF-8 locale passed from lib-gettext.sh + my $is_IS_locale = $ENV{is_IS_locale}; + + my $test = sub { + my ($got, $expect, $msg, $locale) = @_; + # Maybe this system doesn't have the locale we're trying to + # test. + my $locale_ok = setlocale(LC_ALL, $locale); + is(__($got), $expect, "$msg a gettext library + <$locale> locale <$got> turns into <$expect>"); + }; + + my $env_C = sub { + $ENV{LANGUAGE} = 'C'; + $ENV{LC_ALL} = 'C'; + }; + + my $env_is = sub { + $ENV{LANGUAGE} = 'is'; + $ENV{LC_ALL} = $is_IS_locale; + }; + + # Translation's the same as the original + my ($got, $expect) = (('TEST: A Perl test string') x 2); + + if ($has_gettext_library) { + { + local %ENV; $env_C->(); + $test->($got, $expect, "With", 'C'); + } + + { + my ($got, $expect) = ($got, 'TILRAUN: Perl tilraunastrengur'); + local %ENV; $env_is->(); + $test->($got, $expect, "With", $is_IS_locale); + } + } else { + { + local %ENV; $env_C->(); + $test->($got, $expect, "Without", 'C'); + } + + { + local %ENV; $env_is->(); + $test->($got, $expect, "Without", 'is'); + } + } +} diff --git a/t/test-lib.sh b/t/test-lib.sh index ac496aa479..4ae0de8043 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -37,6 +37,7 @@ ORIGINAL_TERM=$TERM # For repeatability, reset the environment to known value. LANG=C LC_ALL=C +LANGUAGE=C PAGER=cat TZ=UTC TERM=dumb @@ -890,6 +891,7 @@ esac test -z "$NO_PERL" && test_set_prereq PERL test -z "$NO_PYTHON" && test_set_prereq PYTHON +test -z "$NO_GETTEXT" && test_set_prereq GETTEXT # test whether the filesystem supports symbolic links ln -s x y 2>/dev/null && test -h y 2>/dev/null && test_set_prereq SYMLINKS diff --git a/upload-pack.c b/upload-pack.c index dc464d78b3..ece9a4b4cd 100644 --- a/upload-pack.c +++ b/upload-pack.c @@ -10,6 +10,7 @@ #include "revision.h" #include "list-objects.h" #include "run-command.h" +#include "gettext.h" static const char upload_pack_usage[] = "git upload-pack [--strict] [--timeout=nn] "; @@ -686,6 +687,8 @@ int main(int argc, char **argv) int i; int strict = 0; + git_setup_gettext(); + git_extract_argv0_path(argv[0]); read_replace_refs = 0; From 02f9c7c96ef2aeb6476d8972e42d7ece15107c4a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=86var=20Arnfj=C3=B6r=C3=B0=20Bjarmason?= Date: Mon, 19 Jul 2010 20:28:11 +0000 Subject: [PATCH 2/9] tests: rename test to work around GNU gettext bug MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rename t0200-gettext.sh to t0200-gettext-basic.sh. Versions of GNU gettext before v0.15 (released in 2006) fail to run the test because of an old bug in gettext.sh. To conform with the FHS gettext.sh prints a help message when it's invoked directly. To do this it checks if $0 is "gettext.sh". This check was broken before v0.15, it was checking for *\gettext.sh (to support Windows), but now correctly checks for *\\gettext.sh. t0200-gettext.sh matched the former broken check, so on systems with an old GNU gettext (like RHEL 5.4) the test simply printed: ./t0200-gettext.sh GNU gettext shell script function library version 0.14.6 Usage: . gettext.sh FATAL: Unexpected exit with code 1 Which is just the gettext.sh help output: $ gettext.sh GNU gettext shell script function library version 0.17 Usage: . gettext.sh Change the test name to t0200-gettext-basic.sh to work around that. Reported-by: Thomas Rast Signed-off-by: Ævar Arnfjörð Bjarmason Tested-by: Thomas Rast Signed-off-by: Junio C Hamano --- t/{t0200-gettext.sh => t0200-gettext-basic.sh} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename t/{t0200-gettext.sh => t0200-gettext-basic.sh} (100%) diff --git a/t/t0200-gettext.sh b/t/t0200-gettext-basic.sh similarity index 100% rename from t/t0200-gettext.sh rename to t/t0200-gettext-basic.sh From 107880a701e9cd42f2a537b7ba74d3cf596a3ebe Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=86var=20Arnfj=C3=B6r=C3=B0=20Bjarmason?= Date: Thu, 12 Aug 2010 22:08:15 +0000 Subject: [PATCH 3/9] gettext: setlocale(LC_CTYPE, "") breaks Git's C function assumptions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Remove the setlocale/LC_CTYPE call from gettext.c, we only need setlocale/LC_MESSAGES to use the message catalog, and setting LC_CTYPE from the environment breaks Git's assumptions about C library functions. Under a non-C locale functions like vsnprintf become locale sensitive, so that they'll e.g. refuse to process ISO-8895-1 data under a UTF-8 locale. This triggered a "your vsnprintf is broken" error on Git's own repository when inspecting v0.99.6~1 under a UTF-8 locale. That commit contains a ISO-8859-1 encoded author name, which the locale aware vsnprintf(3) won't interpolate in the format argument, due to mismatch between the data encoding and the locale. Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Junio C Hamano --- gettext.c | 1 - t/t0203-gettext-setlocale-sanity.sh | 26 ++++++++++++++++++++++++++ 2 files changed, 26 insertions(+), 1 deletion(-) create mode 100755 t/t0203-gettext-setlocale-sanity.sh diff --git a/gettext.c b/gettext.c index 7ae5caed42..db99742bc4 100644 --- a/gettext.c +++ b/gettext.c @@ -17,6 +17,5 @@ extern void git_setup_gettext(void) { } (void)setlocale(LC_MESSAGES, ""); - (void)setlocale(LC_CTYPE, ""); (void)textdomain("git"); } diff --git a/t/t0203-gettext-setlocale-sanity.sh b/t/t0203-gettext-setlocale-sanity.sh new file mode 100755 index 0000000000..a212460081 --- /dev/null +++ b/t/t0203-gettext-setlocale-sanity.sh @@ -0,0 +1,26 @@ +#!/bin/sh +# +# Copyright (c) 2010 Ævar Arnfjörð Bjarmason +# + +test_description="The Git C functions aren't broken by setlocale(3)" + +. ./lib-gettext.sh + +test_expect_success 'git show a ISO-8859-1 commit under C locale' ' + . "$TEST_DIRECTORY"/t3901-8859-1.txt && + test_commit "iso-c-commit" iso-under-c && + git show >out 2>err && + ! test -s err && + grep -q "iso-c-commit" out +' + +test_expect_success GETTEXT_LOCALE 'git show a ISO-8859-1 commit under a UTF-8 locale' ' + . "$TEST_DIRECTORY"/t3901-8859-1.txt && + test_commit "iso-utf8-commit" iso-under-utf8 && + LANGUAGE=is LC_ALL="$is_IS_locale" git show >out 2>err && + ! test -s err && + grep -q "iso-utf8-commit" out +' + +test_done From 44d3770f38f399962946f2a16e492a9d81c4665d Mon Sep 17 00:00:00 2001 From: Thomas Rast Date: Sun, 25 Jul 2010 16:55:46 +0200 Subject: [PATCH 4/9] tests: locate i18n lib&data correctly under --valgrind The new t020[01] for gettext support did not find git-sh-i18n and the translation data when run under --valgrind: lib-gettext.sh tried to locate them under $GIT_EXEC_PATH, which normally is the git build directory, but is changed by --valgrind to point to the wrappers. Introduce a new variable $GIT_BUILD_DIR which can be used to locate data that resides under the build directory, and use that instead. Signed-off-by: Thomas Rast Signed-off-by: Junio C Hamano --- t/lib-gettext.sh | 6 +++--- t/test-lib.sh | 1 + 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/t/lib-gettext.sh b/t/lib-gettext.sh index 831ee38e83..f0cdd3da93 100644 --- a/t/lib-gettext.sh +++ b/t/lib-gettext.sh @@ -5,11 +5,11 @@ . ./test-lib.sh -GIT_TEXTDOMAINDIR="$GIT_EXEC_PATH/share/locale" -GIT_PO_PATH="$GIT_EXEC_PATH/po" +GIT_TEXTDOMAINDIR="$GIT_BUILD_DIR/share/locale" +GIT_PO_PATH="$GIT_BUILD_DIR/po" export GIT_TEXTDOMAINDIR GIT_PO_PATH -. "$GIT_EXEC_PATH"/git-sh-i18n +. "$GIT_BUILD_DIR"/git-sh-i18n if test_have_prereq GETTEXT then diff --git a/t/test-lib.sh b/t/test-lib.sh index 4ae0de8043..08e54386cb 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -768,6 +768,7 @@ else # normal case, use ../bin-wrappers only unless $with_dashes: PATH="$TEST_DIRECTORY/..:$PATH" fi fi +GIT_BUILD_DIR=$(pwd)/.. GIT_TEMPLATE_DIR=$(pwd)/../templates/blt unset GIT_CONFIG GIT_CONFIG_NOSYSTEM=1 From 76c77279b41cad3819e56858882d1d4758f8cb92 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=86var=20Arnfj=C3=B6r=C3=B0=20Bjarmason?= Date: Thu, 19 Aug 2010 16:08:09 +0000 Subject: [PATCH 5/9] test-lib: Use $TEST_DIRECTORY or $GIT_BUILD_DIR instead of $(pwd) and ../ MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Change the redundant calls to $(pwd) to use $TEST_DIRECTORY instead. None of these were being executed after we cd'd somewhere else so they weren't actually needed. This also makes it easier to add support for overriding the test library location and run tests in a different directory than t/. Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Junio C Hamano --- t/test-lib.sh | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/t/test-lib.sh b/t/test-lib.sh index ab4c96788e..9face09c94 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -794,14 +794,14 @@ else # normal case, use ../bin-wrappers only unless $with_dashes: PATH="$TEST_DIRECTORY/..:$PATH" fi fi -GIT_BUILD_DIR=$(pwd)/.. -GIT_TEMPLATE_DIR=$(pwd)/../templates/blt +GIT_BUILD_DIR="$TEST_DIRECTORY"/.. +GIT_TEMPLATE_DIR="$TEST_DIRECTORY"/../templates/blt unset GIT_CONFIG GIT_CONFIG_NOSYSTEM=1 GIT_CONFIG_NOGLOBAL=1 export PATH GIT_EXEC_PATH GIT_TEMPLATE_DIR GIT_CONFIG_NOSYSTEM GIT_CONFIG_NOGLOBAL -. ../GIT-BUILD-OPTIONS +. "$GIT_BUILD_DIR"/GIT-BUILD-OPTIONS if test -z "$GIT_TEST_CMP" then @@ -813,22 +813,22 @@ then fi fi -GITPERLLIB=$(pwd)/../perl/blib/lib:$(pwd)/../perl/blib/arch/auto/Git +GITPERLLIB="$TEST_DIRECTORY"/../perl/blib/lib:"$TEST_DIRECTORY"/../perl/blib/arch/auto/Git export GITPERLLIB -test -d ../templates/blt || { +test -d "$TEST_DIRECTORY"/../templates/blt || { error "You haven't built things yet, have you?" } if test -z "$GIT_TEST_INSTALLED" && test -z "$NO_PYTHON" then - GITPYTHONLIB="$(pwd)/../git_remote_helpers/build/lib" + GITPYTHONLIB="$TEST_DIRECTORY/../git_remote_helpers/build/lib" export GITPYTHONLIB - test -d ../git_remote_helpers/build || { + test -d "$TEST_DIRECTORY"/../git_remote_helpers/build || { error "You haven't built git_remote_helpers yet, have you?" } fi -if ! test -x ../test-chmtime; then +if ! test -x "$TEST_DIRECTORY"/../test-chmtime; then echo >&2 'You need to build test-chmtime:' echo >&2 'Run "make test-chmtime" in the source (toplevel) directory' exit 1 From e4b52593c67d5ce2710158a6bece402fe110cade Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=86var=20Arnfj=C3=B6r=C3=B0=20Bjarmason?= Date: Thu, 19 Aug 2010 16:08:10 +0000 Subject: [PATCH 6/9] test-lib: Use "$GIT_BUILD_DIR" instead of "$TEST_DIRECTORY"/../ MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Change code that used $TEST_DIRECTORY/.. to use $GIT_BUILD_DIR instead, the two are equivalent, but the latter is easier to read. This required moving the assignment od GIT_BUILD_DIR to earlier in the test-lib.sh file. Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Junio C Hamano --- t/test-lib.sh | 29 +++++++++++++++-------------- 1 file changed, 15 insertions(+), 14 deletions(-) diff --git a/t/test-lib.sh b/t/test-lib.sh index 9face09c94..9c089677fb 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -649,7 +649,7 @@ test_create_repo () { repo="$1" mkdir -p "$repo" cd "$repo" || error "Cannot setup test environment" - "$GIT_EXEC_PATH/git-init" "--template=$TEST_DIRECTORY/../templates/blt/" >&3 2>&4 || + "$GIT_EXEC_PATH/git-init" "--template=$GIT_BUILD_DIR/templates/blt/" >&3 2>&4 || error "cannot run git init -- have you built things yet?" mv .git/hooks .git/hooks-disabled cd "$owd" @@ -712,6 +712,8 @@ test_done () { # Test the binaries we have just built. The tests are kept in # t/ subdirectory and are run in 'trash directory' subdirectory. TEST_DIRECTORY=$(pwd) +GIT_BUILD_DIR="$TEST_DIRECTORY"/.. + if test -n "$valgrind" then make_symlink () { @@ -738,7 +740,7 @@ then test -x "$1" || return base=$(basename "$1") - symlink_target=$TEST_DIRECTORY/../$base + symlink_target=$GIT_BUILD_DIR/$base # do not override scripts if test -x "$symlink_target" && test ! -d "$symlink_target" && @@ -757,7 +759,7 @@ then # override all git executables in TEST_DIRECTORY/.. GIT_VALGRIND=$TEST_DIRECTORY/valgrind mkdir -p "$GIT_VALGRIND"/bin - for file in $TEST_DIRECTORY/../git* $TEST_DIRECTORY/../test-* + for file in $GIT_BUILD_DIR/git* $GIT_BUILD_DIR/test-* do make_valgrind_symlink $file done @@ -778,10 +780,10 @@ then elif test -n "$GIT_TEST_INSTALLED" ; then GIT_EXEC_PATH=$($GIT_TEST_INSTALLED/git --exec-path) || error "Cannot run git from $GIT_TEST_INSTALLED." - PATH=$GIT_TEST_INSTALLED:$TEST_DIRECTORY/..:$PATH + PATH=$GIT_TEST_INSTALLED:$GIT_BUILD_DIR:$PATH GIT_EXEC_PATH=${GIT_TEST_EXEC_PATH:-$GIT_EXEC_PATH} else # normal case, use ../bin-wrappers only unless $with_dashes: - git_bin_dir="$TEST_DIRECTORY/../bin-wrappers" + git_bin_dir="$GIT_BUILD_DIR/bin-wrappers" if ! test -x "$git_bin_dir/git" ; then if test -z "$with_dashes" ; then say "$git_bin_dir/git is not executable; using GIT_EXEC_PATH" @@ -789,13 +791,12 @@ else # normal case, use ../bin-wrappers only unless $with_dashes: with_dashes=t fi PATH="$git_bin_dir:$PATH" - GIT_EXEC_PATH=$TEST_DIRECTORY/.. + GIT_EXEC_PATH=$GIT_BUILD_DIR if test -n "$with_dashes" ; then - PATH="$TEST_DIRECTORY/..:$PATH" + PATH="$GIT_BUILD_DIR:$PATH" fi fi -GIT_BUILD_DIR="$TEST_DIRECTORY"/.. -GIT_TEMPLATE_DIR="$TEST_DIRECTORY"/../templates/blt +GIT_TEMPLATE_DIR="$GIT_BUILD_DIR"/templates/blt unset GIT_CONFIG GIT_CONFIG_NOSYSTEM=1 GIT_CONFIG_NOGLOBAL=1 @@ -813,22 +814,22 @@ then fi fi -GITPERLLIB="$TEST_DIRECTORY"/../perl/blib/lib:"$TEST_DIRECTORY"/../perl/blib/arch/auto/Git +GITPERLLIB="$GIT_BUILD_DIR"/perl/blib/lib:"$GIT_BUILD_DIR"/perl/blib/arch/auto/Git export GITPERLLIB -test -d "$TEST_DIRECTORY"/../templates/blt || { +test -d "$GIT_BUILD_DIR"/templates/blt || { error "You haven't built things yet, have you?" } if test -z "$GIT_TEST_INSTALLED" && test -z "$NO_PYTHON" then - GITPYTHONLIB="$TEST_DIRECTORY/../git_remote_helpers/build/lib" + GITPYTHONLIB="$GIT_BUILD_DIR/git_remote_helpers/build/lib" export GITPYTHONLIB - test -d "$TEST_DIRECTORY"/../git_remote_helpers/build || { + test -d "$GIT_BUILD_DIR"/git_remote_helpers/build || { error "You haven't built git_remote_helpers yet, have you?" } fi -if ! test -x "$TEST_DIRECTORY"/../test-chmtime; then +if ! test -x "$GIT_BUILD_DIR"/test-chmtime; then echo >&2 'You need to build test-chmtime:' echo >&2 'Run "make test-chmtime" in the source (toplevel) directory' exit 1 From 07b74a9fc5fe96981c25641366664f188e08d29d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=86var=20Arnfj=C3=B6r=C3=B0=20Bjarmason?= Date: Thu, 19 Aug 2010 16:08:11 +0000 Subject: [PATCH 7/9] test-lib: Allow overriding of TEST_DIRECTORY MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Tests that test the test-lib.sh itself need to be executed in the dynamically created trash directory, so we can't assume $TEST_DIRECTORY is ../ for those. As a side benefit this change also makes it easy for us to move the t/*.sh tests into subdirectories if we ever want to do that. Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Junio C Hamano --- t/test-lib.sh | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/t/test-lib.sh b/t/test-lib.sh index 9c089677fb..804f115bf7 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -711,7 +711,13 @@ test_done () { # Test the binaries we have just built. The tests are kept in # t/ subdirectory and are run in 'trash directory' subdirectory. -TEST_DIRECTORY=$(pwd) +if test -z "$TEST_DIRECTORY" +then + # We allow tests to override this, in case they want to run tests + # outside of t/, e.g. for running tests on the test library + # itself. + TEST_DIRECTORY=$(pwd) +fi GIT_BUILD_DIR="$TEST_DIRECTORY"/.. if test -n "$valgrind" From cc17e9f885e3eee9901f7dab5b6bf7a80b315614 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=86var=20Arnfj=C3=B6r=C3=B0=20Bjarmason?= Date: Thu, 19 Aug 2010 16:08:12 +0000 Subject: [PATCH 8/9] t/t0000-basic.sh: Run the passing TODO test inside its own test-lib MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Change the passing TODO test in t0000-basic.sh to run inside its own test-lib.sh. The motivation is to have nothing out of the ordinary on a normal test run for test smoking purposes. If every normal test run has a passing TODO you're more likely to turn a blind eye to it and not to investigate cases where things really are passing unexpectedly. It also makes the prove(1) output less noisy. Before: All tests successful. Test Summary Report ------------------- ./t0000-basic.sh (Wstat: 0 Tests: 46 Failed: 0) TODO passed: 5 Files=484, Tests=6229, 143 wallclock secs ( 4.00 usr 4.15 sys + 104.77 cusr 351.57 csys = 464.49 CPU) Result: PASS And after: All tests successful. Files=484, Tests=6228, 139 wallclock secs ( 4.07 usr 4.25 sys + 104.54 cusr 350.85 csys = 463.71 CPU) Result: PASS Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Junio C Hamano --- t/t0000-basic.sh | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh index 960208504b..f688bd3ef5 100755 --- a/t/t0000-basic.sh +++ b/t/t0000-basic.sh @@ -54,9 +54,40 @@ test_expect_success 'success is reported like this' ' test_expect_failure 'pretend we have a known breakage' ' false ' + +test_expect_success 'pretend we have fixed a known breakage (run in sub test-lib)' " + mkdir passing-todo && + (cd passing-todo && + cat >passing-todo.sh <out 2>err && + ! test -s err && +cat >expect < Date: Tue, 24 Aug 2010 02:34:10 -0500 Subject: [PATCH 9/9] tests: simplify "missing PREREQ" message MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit When a test has no prerequisites satisfied (the usual case), instead of "missing THING of THING", just say "missing THING". This does not affect the output when a test is skipped due to a missing prerequisites if another prerequisite is satisfied. For example: instead of ok 8 # skip notes work (missing EXPENSIVE of EXPENSIVE) ok 9 # skip notes timing with /usr/bin/time (missing EXPENSIVE of USR_BIN_TIME,EXPENSIVE) write ok 8 # skip notes work (missing EXPENSIVE) ok 9 # skip notes timing with /usr/bin/time (missing EXPENSIVE of USR_BIN_TIME,EXPENSIVE) Cc: Ævar Arnfjörð Bjarmason Signed-off-by: Jonathan Nieder Signed-off-by: Junio C Hamano --- t/test-lib.sh | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/t/test-lib.sh b/t/test-lib.sh index 804f115bf7..c67a9c6e6d 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -418,8 +418,14 @@ test_skip () { fi case "$to_skip" in t) + of_prereq= + if test "$missing_prereq" != "$prereq" + then + of_prereq=" of $prereq" + fi + say_color skip >&3 "skipping test: $@" - say_color skip "ok $test_count # skip $1 (missing $missing_prereq of $prereq)" + say_color skip "ok $test_count # skip $1 (missing $missing_prereq${of_prereq})" : true ;; *)