View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0000937 | luatex | luatex bug | public | 2015-05-11 18:32 | 2015-11-02 17:40 |
Reporter | pdfkungfoo | Assigned To | Hans Hagen | ||
Priority | normal | Severity | minor | Reproducibility | always |
Status | closed | Resolution | won't fix | ||
Platform | Tex Live 2014 | OS | OS X Mavericks | OS Version | 10.9.5 |
Product Version | 0.80.1 | ||||
Target Version | maybe never (> v 1) | ||||
Summary | 0000937: lualatex trips over some JPEGS and aborts PDF generating process (or leads to unexpected results) | ||||
Description | The `lualatex` command trips over some JPEG files and aborts the PDF generating process. Here is an MWE, `mini.tex`: ############################## \documentclass[a4paper]{article} \usepackage{graphicx} \begin{document} \thispagestyle{empty} \begin{figure}[htbp] \centering \includegraphics{./testwiz.jpg} \end{figure} \end{document} ############################## The `testwiz.jpg` was generated with ImageMagick from its built-in `wizard:` image: convert wizard: -frame 1 testwiz.jpg As it happens, this JPEG now has the following JFIF headers set, as seen in a hex editor: +-------------+-------------------------+------------+------------+ | Byte-Offset | 0x0d | 0x0e | 0x10 | +-------------+-------------------------+------------+------------+ | Field-Name | "Units" | "XDensity" | "YDensity" | +-------------+-------------------------+------------+------------+ | Value | 1 | 1 | 1 | +-------------+-------------------------+------------+------------+ | Meaning | "X/YDensities are DPI" | 1 dpi | 1 dpi | +-------------+-------------------------+------------+------------+ Exiftool reports this: exiftool testwiz.jpg Resolution Unit : inches X Resolution : 1 Y Resolution : 1 Running `lualatex mini.tex` yields this: LuaTeX warning: arithmetic: number too big ! Dimension too large. <recently read> \dimen@ l.7 \includegraphics{./testwiz.jpg} ? Hitting [Enter] repeatedly completes the PDF generation process. The resulting document has 66 kBytes. It shows only a white page in the viewer. Unpacking the PDF: qpdf --qdf --object-streams=disable mini.pdf unpacked-mini.pdf Inspecting `unpacked-mini.pdf` in a text editor shows these points: 1. The image data is embedded and complete. It is in PDF object no. 8, containing 65456 Bytes of JPEG data in the object's stream. 1. The `/Contents` object is no. 5 which contains this line of PDF source code: -32645.5776 0 0 -32645.5776 124.802 706.129 cm /Im1 Do This means the image is displaced and scaled by a huge amount and is outside the normal page `/MediaBox`. After changing above line to 481 0 0 640 124.802 76.093 cm /Im1 Do (be careful to ***overwrite*** the old bytes, and not add or subtract from the total number of bytes!) and saving the modification the image will become visible on the PDF page. I think it is stupid for `lualatex` to rely on the values of the JFIF header for scaling the image in (absence of explicit user provided TeX commands giving orders about the wanted scaling) when the headers says the resolution is 1 DPI: * Almost always a resolution of DPI = 1 does not make sense. * Almost always the resulting image on the page comes out in un-expected ways. Imagine a user with two identically looking files, where one file "works" on a PDF page and looks as expected while the other one exhibits the described behavior just because of one Bit being flipped in the JFIF header! You can manipulate the JFIF headers of a JPEG without changing any image pixels with the help of `exiftool`: exiftool \ -jfif:Xresolution=300 \ -jfif:Yresolution=300 \ -jfif:ResolutionUnit=inches \ the.jpg Setting `-jfif:ResolutionUnit=cm` sets the "Units" field to "centimeters per inch". Setting `-jfif:ResolutionUnit=None` sets the "Units" field to "Xdensity and Ydensity are pixel ratios". The following settings in the JFIF header all currently cause different PDF output with the `mini.tex` input file: +-------------+-----------------------------+------------+------------+ | Byte-Offset | 0x0d | 0x0e | 0x10 | +-------------+-----------------------------+------------+------------+ | Field-Name | "Units" | "XDensity" | "YDensity" | +-------------+-----------------------------+------------+------------+ | Value | 1 | 1 | 1 | +-------------+-----------------------------+------------+------------+ | Value | 1 | 7 | 7 | +-------------+-----------------------------+------------+------------+ | Value | 1 | 100 | 100 | +-------------+-----------------------------+------------+------------+ | Value | 1 | 200 | 200 | +-------------+-----------------------------+------------+------------+ | Value | 1 | 300 | 300 | +-------------+-----------------------------+------------+------------+ | Value | 1 | 100 | 300 | +-------------+-----------------------------+------------+------------+ | Value | 1 | 300 | 100 | +-------------+-----------------------------+------------+------------+ | Value | 1 | 300 | 1 | +-------------+-----------------------------+------------+------------+ | Meaning | "X/YDensities are DPI" | <above> px | <above> px | +-------------+-----------------------------+------------+------------+ At least the behavior here is somehow consistent: the scaling, buckling and stretching happens according to the DPI settings. But if the settings are changed to these: +-------------+-----------------------------+------------+------------+ | Byte-Offset | 0x0d | 0x0e | 0x10 | +-------------+-----------------------------+------------+------------+ | Field-Name | "Units" | "XDensity" | "YDensity" | +-------------+-----------------------------+------------+------------+ | Value | 0 | 7 | 7 | +-------------+-----------------------------+------------+------------+ | Value | 0 | 100 | 100 | +-------------+-----------------------------+------------+------------+ | Value | 0 | 200 | 200 | +-------------+-----------------------------+------------+------------+ | Value | 0 | 300 | 300 | +-------------+-----------------------------+------------+------------+ | Value | 0 | 100 | 300 | +-------------+-----------------------------+------------+------------+ | Value | 0 | 300 | 100 | +-------------+-----------------------------+------------+------------+ | Value | 0 | 300 | 1 | +-------------+-----------------------------+------------+------------+ | Meaning | "X/YDensities pixel ratios" | <above> px | <above> px | +-------------+-----------------------------+------------+------------+ then the output is identical for each case, not changing the pixel aspect ratios as it should. So... 1. `lualatex` does not respect the "X/Ydensity" JFIF fields if "Units" is set to "0", meaning "X/YDensities are pixel ratios". 2. `lualatex` does ONLY respect the "X/Ydensity" JFIF fields if "Units" is set to "1", meaning "X/YDensities are DPI". This is inconsistent. If it creates different output in case 2 it should also do so in case 1. I even think the `lualatex` PDF producing utility should NOT use the JFIF header metadata AT ALL for embedding JPEG images into pages. Because: * Most users are unaware of these JFIF headers. * Most users don't know how to change these headers. * These headers control the scaling of JPEGs on PDF pages only indirectly. * Most users expect to control the scaling of images in PDF output by different (LaTeX-provided) means directly. * The treatment of the various JFIF headers is inconsistent: - if set as "X/Densities are DPI" they are NOT respected; - if set as "X/YDensities are pixel ratios" they are (somehow) respected. # Executive summary 1. `lualatex` behaves unreasonable when it generates PDFs which include JPEG images: - On the one hand, "XDensity" and "YDensity" fields are completely IGNORED if they describe "DPI" (or "DPcm"). - On the other hand, "XDensity" and "YDensity" fields ARE evaluated if they describ "pixel aspect ratios", but the outcome is totally unexpected for most users. 2. `lualatex` should ignore the JFIF fields completely and only obey the statements given by the document's LaTeX code. In the absence of such code, the PDF generators should assume 72 DPI. ---- | ||||
Additional Information | This is the first time I report on Mantis. I don't know if my writeup and my tables will come out readable or not. If not, my apologies already now. Please also refer to these resources: 1. http://tex.stackexchange.com/q/244222/8747 2. http://tex.stackexchange.com/q/243753/8747 3. http://www.imagemagick.org/discourse-server/viewtopic.php?f=3&t=27589 | ||||
Tags | No tags attached. | ||||
|
|
|
What a crappy user interface to use is this bug tracker here! For a willing and well-meaning user to have to deal with this is a total pain in the a*#. 1. Can't you at least use a mono-spaced font?! 2. And tell it not to eat up multiple spaces! 3. And give me a preview of what I've written before I finally submit? 4. And let me edit my posting after I discover a typo or an error?! 5. [... no, I'm not wasting my time any more...] As it stands, this will be used by me again... ...only in extreme emergencies! |
|
Thank you for the report. We will investigate asap. Can you check against the latest 0.80.0 ? |
|
Luigi, I don't know how to check against 0.80.0. I totally rely on the packages the "Tex Live Utility" gives me (which I update almost daily). But if there isn't a *totally easy* and *easily revertible* way to update to 0.80.0 I'm not willing to jump through loops here. (I'm already handling other self-compiled software packages on my notebook. But these I *know* how to deal with in case of problems -- everything LaTeX/TeX related I have no clue. I only came to LaTeX/LuaLaTeX via `pandoc`...) |
|
Confirmed with 0.80.1 (TeX Live 2015) (rev 5241) |
|
|
|
mini.log (6,615 bytes)
This is LuaTeX, Version beta-0.80.1 (TeX Live 2015) (rev 5241) (format=lualatex 2015.5.8) 11 MAY 2015 19:17 restricted \write18 enabled. **./mini.tex (./mini.tex LaTeX2e <2014/05/01> Babel <3.9l> and hyphenation patterns for 53 languages loaded. No auxiliary output files. (/opt/tex/texlive/2014/texmf-dist/tex/latex/base/minimal.cls Document Class: minimal 2001/05/25 Standard LaTeX minimal class ) (/opt/tex/texlive/2014/texmf-dist/tex/latex/graphics/graphicx.sty Package: graphicx 2014/10/28 v1.0g Enhanced LaTeX Graphics (DPC,SPQR) (/opt/tex/texlive/2014/texmf-dist/tex/latex/graphics/keyval.sty Package: keyval 2014/10/28 v1.15 key=value parser (DPC) \KV@toks@=\toks14 ) (/opt/tex/texlive/2014/texmf-dist/tex/latex/graphics/graphics.sty Package: graphics 2014/10/28 v1.0p Standard LaTeX Graphics (DPC,SPQR) (/opt/tex/texlive/2014/texmf-dist/tex/latex/graphics/trig.sty Package: trig 1999/03/16 v1.09 sin cos tan (DPC) ) (/opt/tex/texlive/2014/texmf-dist/tex/latex/latexconfig/graphics.cfg File: graphics.cfg 2010/04/23 v1.9 graphics configuration of TeX Live ) Package graphics Info: Driver file: pdftex.def on input line 91. (/opt/tex/texlive/2014/texmf-dist/tex/latex/pdftex-def/pdftex.def File: pdftex.def 2011/05/27 v0.06d Graphics/color for pdfTeX (/opt/tex/texlive/2014/texmf-dist/tex/generic/oberdiek/infwarerr.sty Package: infwarerr 2010/04/08 v1.3 Providing info/warning/error messages (HO) ) (/opt/tex/texlive/2014/texmf-dist/tex/generic/oberdiek/ltxcmds.sty Package: ltxcmds 2011/11/09 v1.22 LaTeX kernel commands for general use (HO) ) \Gread@gobject=\count79 )) \Gin@req@height=\dimen102 \Gin@req@width=\dimen103 ) No file mini.aux. LaTeX Font Info: Checking defaults for OML/cmm/m/it on input line 4. LaTeX Font Info: ... okay on input line 4. LaTeX Font Info: Checking defaults for T1/cmr/m/n on input line 4. LaTeX Font Info: ... okay on input line 4. LaTeX Font Info: Checking defaults for OT1/cmr/m/n on input line 4. LaTeX Font Info: ... okay on input line 4. LaTeX Font Info: Checking defaults for OMS/cmsy/m/n on input line 4. LaTeX Font Info: ... okay on input line 4. LaTeX Font Info: Checking defaults for OMX/cmex/m/n on input line 4. LaTeX Font Info: ... okay on input line 4. LaTeX Font Info: Checking defaults for U/cmr/m/n on input line 4. LaTeX Font Info: ... okay on input line 4. (/opt/tex/texlive/2014/texmf-dist/tex/context/base/supp-pdf.mkii [Loading MPS to PDF converter (version 2006.09.02).] \scratchcounter=\count80 \scratchdimen=\dimen104 \scratchbox=\box26 \nofMPsegments=\count81 \nofMParguments=\count82 \everyMPshowfont=\toks15 \MPscratchCnt=\count83 \MPscratchDim=\dimen105 \MPnumerator=\count84 \makeMPintoPDFobject=\count85 \everyMPtoPDFconversion=\toks16 ) (/opt/tex/texlive/2014/texmf-dist/tex/generic/oberdiek/pdftexcmds.sty Package: pdftexcmds 2011/11/29 v0.20 Utility functions of pdfTeX for LuaTeX (HO) (/opt/tex/texlive/2014/texmf-dist/tex/generic/oberdiek/ifluatex.sty Package: ifluatex 2010/03/01 v1.3 Provides the ifluatex switch (HO) Package ifluatex Info: LuaTeX detected. ) (/opt/tex/texlive/2014/texmf-dist/tex/generic/oberdiek/ifpdf.sty Package: ifpdf 2011/01/30 v2.3 Provides the ifpdf switch (HO) Package ifpdf Info: pdfTeX in PDF mode is detected. ) Package pdftexcmds Info: \pdf@primitive is available. Package pdftexcmds Info: \pdf@ifprimitive is available. Package pdftexcmds Info: \pdfdraftmode found. (/opt/tex/texlive/2014/texmf-dist/tex/generic/oberdiek/luatex-loader.sty Package: luatex-loader 2010/03/09 v0.4 Lua module loader (HO) (/opt/tex/texlive/2014/texmf-dist/scripts/oberdiek/oberdiek.luatex.lua)) \pdftexcmds@toks=\toks17 ) (/opt/tex/texlive/2014/texmf-dist/tex/latex/oberdiek/epstopdf-base.sty Package: epstopdf-base 2010/02/09 v2.5 Base part for package epstopdf (/opt/tex/texlive/2014/texmf-dist/tex/latex/oberdiek/grfext.sty Package: grfext 2010/08/19 v1.1 Manage graphics extensions (HO) (/opt/tex/texlive/2014/texmf-dist/tex/generic/oberdiek/kvdefinekeys.sty Package: kvdefinekeys 2011/04/07 v1.3 Define keys (HO) )) (/opt/tex/texlive/2014/texmf-dist/tex/latex/oberdiek/kvoptions.sty Package: kvoptions 2011/06/30 v3.11 Key value format for package options (HO) (/opt/tex/texlive/2014/texmf-dist/tex/generic/oberdiek/kvsetkeys.sty Package: kvsetkeys 2012/04/25 v1.16 Key value parser (HO) (/opt/tex/texlive/2014/texmf-dist/tex/generic/oberdiek/etexcmds.sty Package: etexcmds 2011/02/16 v1.5 Avoid name clashes with e-TeX commands (HO) ))) Package grfext Info: Graphics extension search list: (grfext) [.png,.pdf,.jpg,.mps,.tif,.jpeg,.PNG,.PDF,.JPG,.JPEG,.eps] (grfext) \AppendGraphicsExtensions on input line 452. (/opt/tex/texlive/2014/texmf-dist/tex/latex/latexconfig/epstopdf-sys.cfg File: epstopdf-sys.cfg 2010/07/13 v1.3 Configuration of (r)epstopdf for TeX Live )) LuaTeX warning (file ./testwiz.jpg): The image specifies an unusual resolution o f 1dpi by 1dpi. LuaTeX warning: arithmetic: number too big LuaTeX warning: arithmetic: number too big ! Dimension too large. <recently read> \dimen@ l.5 \includegraphics{./testwiz.jpg} ? s OK, entering \scrollmode... <./testwiz.jpg, id=1, --32768.0pt x 0.0pt> ! Dimension too large. <to be read again> \advance l.5 \includegraphics{./testwiz.jpg} I can't work with sizes bigger than about 19 feet. Continue and I'll use the largest value I can. File: ./testwiz.jpg Graphic file (type jpg) <use ./testwiz.jpg> Package pdftex.def Info: ./testwiz.jpg used on input line 5. (pdftex.def) Requested size: 16383.99998pt x 0.0pt. LuaTeX warning: arithmetic: number too big LuaTeX warning: arithmetic: number too big Overfull \hbox (15934.245pt too wide) in paragraph at lines 5--6 [][][] [] [1<./testwiz.jpg LuaTeX warning: The image specifies an unusual resolution of 1dpi by 1dpi. >]) Here is how much of LuaTeX's memory you used: 1404 strings out of 494721 100000,89155 words of node,token memory allocated 240 words of node memory still in use: 2 hlist, 1 vlist, 1 rule, 2 glue, 34 glue_spec, 1 write nodes avail lists: 2:12,3:1,4:15,6:10,7:1,9:11 5049 multiletter control sequences out of 65536+600000 15 fonts using 601983 bytes 37i,4n,23p,215b,102s stack positions out of 5000i,500n,10000p,200000b,100000s Output written on mini.pdf (1 page, 66532 bytes). PDF statistics: 9 PDF objects out of 1000 (max. 8388607) 4 compressed objects within 1 object stream 0 named destinations out of 1000 (max. 131072) 1 words of extra memory for PDF output out of 10000 (max. 10000000) |
|
exiftools 9.46 with your image says : Resolution Unit : inches X Resolution : 1 Y Resolution : 1 : Image Size : 482x642 The image size is hence 482in x 642in, and both are bigger than 2^16pt (1pt =1/72.27 in) which is the max dimension accepted by the TeX engines pdftex, xetex, luatex. In this case the image is not loaded (in luatex) even if the user specifies a transformation matrix that gives an acceptable final dimensions. Note that 1dpi is not wrong by itself --- 20x20 at 1dpi is perfectly fine --- the limits are on the metric dimensions. Anyway, from luatex 0.80.0 there is a warning for 1dpi: LuaTeX warning: The image specifies an unusual resolution of 1dpi by 1dpi. LuaTeX warning: arithmetic: number too big LuaTeX warning: arithmetic: number too big |
|
@LuigiScarso: I know that the image size is 482x642 pixels, and a 1 DPI JFIF setting, if respected, will create an image of 482in x 642in. That is exactly my argument: scaling the image based on a (most likely) very completely uncontrolled JFIF header setting is not a good idea. The changed v0.80.0 warning does not help much, sorry. If the lualatex behavior does not change, it will still not create a visible image on the PDF page. |
|
The image is valid, and as I have said there are 1dpi images that are perfectly fine with luatex. To put the box with the image on the main vertical list, Luatex needs to know its metric dimensions and the only way, when the dpi is known, is to multiply the dpi for the width and the height. In this case the dimensions are over 2^16pt, and the box cannot be built. It is supposed that a user knows these features, checking if an image fits into this limit, and acting as consequence if this is not the case. We agree that it's desirable that a transformation matrix could act before the calculation of the dimensions, but this doesn't solve the problem, as also impose a scaling to 2^16pt . This is not a bug, but a feature that could be improved in a next release. |
|
Ah, sorry, I meant 32768pt = 2^15pt not 2^16pt. |
|
Also the message LuaTeX warning: arithmetic: number too big comes from scaled ext_xn_over_d(scaled x, scaled n, scaled d) { double r = (((double) x) * ((double) n)) / ((double) d); if (r > DBL_EPSILON) r += 0.5; else r -= 0.5; if (r >= (double) max_integer || r <= -(double) max_integer) luatex_warn("arithmetic: number too big"); return (scaled) r; } where #define max_integer 0x7FFFFFFF so it's related to a value that cannot fit into a scaled (which is defined as int). Even if the image fits into a scaled, it can be still problematic because the \maxdimen is 16383.99998pt ~= 2^14pt . |
|
it's dangerous to change behaviour of the inclusion and interpretation of fields without much testing, we keep an eye on it but zero or one pixel images are ot common in tex related workflows |
|
we will not touch the inclusion code; this issue relates to the maximum dimension that tex can handle and 1 dpi is resulting in an overflow ... so be it (LS/HH) |
Date Modified | Username | Field | Change |
---|---|---|---|
2015-05-11 18:32 | pdfkungfoo | New Issue | |
2015-05-11 18:32 | pdfkungfoo | File Added: testwiz.jpg | |
2015-05-11 18:43 | pdfkungfoo | Note Added: 0001365 | |
2015-05-11 18:51 | luigi scarso | Note Added: 0001366 | |
2015-05-11 19:04 | pdfkungfoo | Note Added: 0001367 | |
2015-05-11 19:18 | oneiros | Note Added: 0001368 | |
2015-05-11 19:18 | oneiros | File Added: mini.tex | |
2015-05-11 19:18 | oneiros | File Added: mini.log | |
2015-05-11 19:20 | oneiros | Product Version | 0.79.1 => 0.80.1 |
2015-05-11 20:34 | luigi scarso | Note Added: 0001369 | |
2015-05-12 00:59 | pdfkungfoo | Note Added: 0001370 | |
2015-05-12 02:21 | luigi scarso | Note Added: 0001371 | |
2015-05-12 08:40 | luigi scarso | Note Added: 0001372 | |
2015-05-12 09:07 | luigi scarso | Note Added: 0001373 | |
2015-10-23 09:07 | Hans Hagen | Note Added: 0001473 | |
2015-10-23 09:10 | Hans Hagen | Target Version | => maybe never (> v 1) |
2015-10-24 21:01 | Hans Hagen | Status | new => acknowledged |
2015-11-02 17:40 | Hans Hagen | Note Added: 0001495 | |
2015-11-02 17:40 | Hans Hagen | Status | acknowledged => closed |
2015-11-02 17:40 | Hans Hagen | Assigned To | => Hans Hagen |
2015-11-02 17:40 | Hans Hagen | Resolution | open => won't fix |