:: -------------------------------------------------------------
:: libstartup.txt  --  documentation for the libstartup tool
::
:: History:
::
:: 24-Sep-2007 Eero Tamminen
:: - Update for Maemo Chinook
::
:: 30-Mar-2005 Simo Piiroinen
:: - documented custom timestamp generation
::
:: 27-Jan-2005 Simo Piiroinen
:: - rewritten from scratch
:: -------------------------------------------------------------

::HR
		     LIBSTARTUP User Guide

		   27-Jan-2005 Simo Piiroinen
			    (draft)
::HR

Contents:
::TOC

Abstract
========

The libstartup is a LD_PRELOAD binary used for measuring startup times
for Gtk applications.


Gotchas
=======

Use of LD_PRELOAD has some shortcomings:
- It can be used only if the applications can be run directly.
  I.e. it doesn't work with maemo-launched Gtk applications.
- Prelink information cannot be used with LD_PRELOAD.  This means that
  prelinked applications spend time in resolving library symbols when
  this is used.

These make libstartup fairly useless for measuring startup times for
the pre-installed applications of N770 and N800.

For those, the use of the complimentary "xresponse" tool is recommend
instead.  It cannot give break down of the startup time, but on the
other hand it measures time from invoking the application from the UI
until it has finished all screen updates.

NOTE: In Scratchbox LD_PRELOAD doesn't work directly because Scratchbox
uses that by itself, use the "run-with-startup-time" script instead.


Installation
============

The libstartup.so is part of "sp-startup-time" Debian package.


How does it work
================

The libstartup works by Linux dynamic loader preload feature.
Few key functions typically called during startup of GTK
applications are intercepted and call time stamps collected. A
GTK idle callback is also installed and used to determine when
the startup has finished.

If the application uses gtk functions and reaches idle state a
entry will be immediately written to syslog.

If instructed to do so (via environment, see below) libstartup
will also write full log of intercepted calls. Unlike the
syslog writing, this is done at the time when the application
is unloaded.

How to use it
=============

Preloading
----------

You can use two methods for preloading the libstartup binary
during application execution: environment or global ld.so
configuration.

Setting the LD_PRELOAD environment variable like below is
preferred as that way it doesn't affect any other programs:

   $ LD_PRELOAD=/usr/lib/sp-startup-time/libstartup.so app-to-run

However if the "application-to-run" happens to be a launcher
script, not a real binary you might need to edit the script
itself to contain the preload setting like this.

   #!/bin/sh
   export LD_PRELOAD=/usr/lib/sp-startup-time/libstartup.so
   app-to-run


Apart from the environment you can use the global ld.so
configuration to force preloading libstartup for all
applications by editing the /etc/ld.so.preload file. This
is not however encouraged as any mistake might make the
system unbootable.


Configuration
-------------

Normally libstartup writes the time application used to reach
Gtk idle into syslog, but you can get more detailed information
and direct it also to elsewhere with something like this:

   export STARTUP_IDLE_QUIT=yes
   export STARTUP_WRITE_LOG=yes
   export STARTUP_LOG_FILE=/home/user/startup-log.csv

See also "How to interpret the results" below.


The environment variables and how they control the runtime
behavior of libstartup are:

STARTUP_LOG_DIR

	Explicitly set the directory where libstartup will
	write the logs.
	
	If not set the logs will be written to the current
	working directory of the process at the time when
	libstartup init is called - that is just before the
	main function is entered.


STARTUP_LOG_FILE

	Explicitly set path used for writing logs.
	
	If not set <$STARTUP_LOG_DIR>/<appname>--<pid>.startup
	will be used.

STARTUP_WRITE_LOG
	
	If set to 'yes', and the process reaches gtk idle
	state, the full startup log will be written.
	
	If set to 'force' the startup log will be written
	always, not just for gtk apps.

STARTUP_IDLE_QUIT

	If set (to anything) gtk_main_quit() will be called
	when idle state is reached.
	
	Useful for batch testing several runs / applications.
	

The values can also be set by writing them to
"/tmp/startup.env". This allows changing configuration for
applications for which the environment values would be hard to
modify. You could for example use "/etc/ld.so.preload" to
globally enable libstartup use for all applications and then
set the configuration options via "/tmp/startup.env".


Custom timestamps
-----------------

Sometimes you might want to add application specific
measurement points to the libstartup log. This can be
accomplished by inserting code similar to the example
below to the application source file(s):

    +------- hello.c -------------------------------------------
    |   1: #include <stdio.h>
    |   2: 
    |   3: void __cyg_profile_func_enter(const void *, const void *);
    |   4: 
    |   5: #define libstartup_custom_timestamp(text) \
    |   6:   __cyg_profile_func_enter((void *)(-1),text)
    |   7: 
    |   8: int main(void)
    |   9: {
    |  10:   sleep(1);
    |  11: 
    |  12:   libstartup_custom_timestamp("main1");
    |  13:   printf("hello, "); fflush(stdout);
    |  14:   sleep(1);
    |  15: 
    |  16:   libstartup_custom_timestamp("main2");
    |  17:   printf("world\n"); fflush(stdout);
    |  18:   sleep(1);
    |  19: 
    |  20:   libstartup_custom_timestamp("main3");
    |  21: 
    |  22:   sleep(1);
    |  23:   return 0;
    |  24: }
    +------- hello.c -------------------------------------------


It compiles as usual:

    % gcc -o hello hello.c
    
It executes as normal without libstartup:

    % ./hello 
    hello, world

And you get the custom timestamps with:

    % STARTUP_WRITE_LOG=f LD_PRELOAD=.../libstartup.so ./hello
    hello, world


    +------- hello--5092.startup -------------------------------
    |   1: generator=sp-startup-time 0.0.7
    |   2: create_tod=1195658552.962703
    |   3: 
    |   4: t_tod,t_real,t_user,t_sys,f_real,f_user,f_sys,name
    |   5: 1195658552.962703,0.000,0.000,0.000,0.010,0.000,0.000,create
    |   6: 1195658552.972705,0.010,0.000,0.000,0.000,0.000,0.000,lib_init
    |   7: 1195658553.980551,1.020,0.000,0.000,0.000,0.000,0.000,main1
    |   8: 1195658554.985998,2.020,0.000,0.000,0.000,0.000,0.000,main2
    |   9: 1195658555.991298,3.020,0.000,0.000,0.000,0.000,0.000,main3
    |  10: 1195658556.996472,4.030,0.000,0.000,0.000,0.000,0.000,lib_exit
    |  11: 1195658556.996473,4.030,0.000,0.000,0.000,0.000,0.000,lib_fini
    +------- hello--5092.startup -------------------------------


The rationale behind hijacking __cyg_profile_func_enter():

- we can't override functions defined in the main application
  binary using LD_PRELOAD -> it must be in some library
  
- we could provide dummy library with some kind of hook, but
  that would require modifying the application linking, not
  just the source code
  
-> we use a dummy function provided by gnu libc

Note: While this does not prevent using libtimes together with
      libstartup, try to avoid it. If you decide to use the two
      libraries together, you MUST specify LD_PRELOAD order so
      that libstartup.so is before libtimes.so.


Note: The amount of measurement points is currently limited to
      32. For GTK applications this should leave 20 or so
      custom entries.

Note: If the same custom tag is used several times, only the
      last occurence is placed to the log file.


How to interpret results
========================

syslog
------

+------- syslog --------------------------------------------
|  34: Jan 27 14:33:51 localhost libstartup[12706]: firefox-bin=1.19 seconds
|  35: Jan 27 14:34:32 localhost libstartup[12727]: firefox-bin=1.13 seconds
+------- syslog --------------------------------------------

Elapsed wall-clock time from process creation to gtk idle time
is written as seconds.


startup log
-----------

Entry time stamp:

- t_tod  gettimeofday() compatible timestamp
- t_real elapsed time since process create
- t_user CPU time spent in user land before call
- t_sys  CPU time spent in kernel space before call
 
Time within:

- f_real elapsed time within entry
- t_user CPU time spent in user land during the call
- t_sys  CPU time spent in kernel space during the call
  
Entry identification:

- name   name of called function, or
         function1.leave->function2.entry transition

+------- firefox-bin--12727.startup ------------------------
|   1: generator=sp-startup-time 0.0.7
|   2: create_tod=1195658552.962703
|   3: 
|   4: t_tod,t_real,t_user,t_sys,f_real,f_user,f_sys,name
|   5: 1195658611.448291,0.000,0.000,0.000,0.280,0.000,0.010,create
|   6: 1195658611.728293,0.280,0.000,0.010,0.000,0.000,0.000,lib_init
|   7: 1195658611.729729,0.280,0.000,0.010,0.000,0.000,0.000,gtk_init
|   8: 1195658611.735175,0.280,0.000,0.010,3.010,0.110,0.040,gtk_init->gtk_main
|   9: 1195658614.744454,3.290,0.110,0.050,0.000,0.000,0.000,gtk_main
|  10: 1195658614.744454,3.290,0.110,0.050,2.290,0.290,0.030,gtk_main->gui_idle
|  11: 1195658617.032341,5.580,0.400,0.080,0.000,0.000,0.000,gui_idle
|  12: 1195658617.032341,5.580,0.400,0.080,18.740,0.120,0.020,gui_idle->gtk_main_quit
|  13: 1195658635.775691,24.320,0.520,0.100,0.010,0.000,0.000,gtk_main_quit
|  14: 1195658635.775702,24.330,0.520,0.100,0.080,0.050,0.000,gtk_main_quit->lib_exit
|  15: 1195658635.855718,24.410,0.570,0.100,0.000,0.000,0.000,lib_exit
|  16: 1195658635.855720,24.410,0.570,0.100,0.000,0.000,0.000,lib_fini
+------- firefox-bin--12727.startup ------------------------



The following calls are intercepted and logged. The entries
with preceding '+' will have time stamps for both entry to
and leave from function. The ones marked with '-' will have
only entry time stamps.

+ create	
+ lib_init	(installs lib_exit as atexit handler)
+ gtk_init	(installs gui_idle as gtk idle handler)
- gtk_main
- gui_idle
+ gtk_main_quit
- lib_exit	
- lib_fini	

The timestamps are obtained with times() syscall which allows
us to trace:

- real time (elapsed wall clock time)
- user time (time spent executing code)
- sys time  (time spent in kernel servicing the process)


If there time difference between leaving the previous traced
function and entering the next, a transition entry will be
written. The interpretation of these transition entries depends
on the situation, but generally:

lib_init -> gtk_init

	Usually libstartup initialization is executed just before
	control is yielded to main() entry of the application.
	So this transition can be considered as "non-gui-init".

gtk_init -> gtk_main

	Setting up application main GUI.
	
	
gtk_main -> gui_idle

	Realizing application main GUI.
	
gui_idle -> gtk_main_quit

	Normal application operation.
	
gtk_main_quit -> lib_exit

	Application shutdown
	
lib_exit -> lib_fini

	As the lib_exit is most likely the last atexit handler
	called and lib_fini the first library unload code
	executed, this is very unlikely to have nonzero time.
	
	
The "create" entry needs a bit of explanation too. The idea is
that this contains time taken by the dynamic linker to bind the
various binaries to the process. As the init code from
libstartup is if not the last at least one of the last things
done before entering application main() function, we can guess
that the CPU time spent before this is from dynamic loader.
Perhaps surprisingly determining the real time is quite
difficult. The reason is that the syscalls that can be used to
query time values return corrected real time and the process
create time (as available via /proc/pid/status) contains
uncorrected jiffies value. To overcome this the libstartup
forks a child process and uses the create time difference
between parent and child as elapsed real time estimate. The
child process is terminated right after the time difference
evaluation.


The idle detection is also not without problems. It is common
that there is a brief idle period right after entering gtk_main.
For this reason the idle status is is noted only when the
idle handler is called enough times without using CPU for
other purposes (in the context of application process).

