Learning pthreads

Q 1 How to create Thread , Get status and  Synchronize ?
Ans Get Status :
  1) thread must call pthread_join(threads[t] ,&status); where  void *status = NULL;
       this status can be retrieved by caller    
       printf("Main: completed join with thread  having a status   of %lu\n",(long)status);
      Don t fetch *status  it will core dump 
Thread creator must call either  pthread_join or pthread_detach() [this is applicable if thread is not created as Attribute detachable by default threads are non detachable ]

pthread_detach() function indicates that system resources for the specified thread should be reclaimed when the thread ends. pthread_detach() routine can be used to explicitly detach a thread even though it was created as joinable. After pthread_detach() has been issued, it is not valid to try to pthread_join() with the target thread.
----- The pthread_join() function waits for a thread to terminate, detaches the thread, then returns the threads exit status. if it was specified in the target thread's call to pthread_exit().

If thread resources are not freed even if threads are complete resources will not get freed and after 200-250 threads creation errorno 11 will be return "Resource temporarily unavailable".

2) Thread can give status back by return or  pthread_exit(); both are same
Note : When a joinable thread terminates, its memory resources (thread

descriptor and stack) are not deallocated until another thread performs
       pthread_join on it. Therefore, pthread_join must  be  called  once  for
       each joinable thread created to avoid memory leaks.

man of pthread_exit()

The  pthread_exit()  function terminates the calling thread and returns a value via retval that (if
       the  thread  is  joinable)  is  available  to  another  thread  in  the  same  process  that  calls
       pthread_join(3).
       Any  clean-up  handlers  established  by pthread_cleanup_push(3) that have not yet been popped, are
       popped (in the reverse of the order in which they were pushed) and executed.  If the thread has any
       thread-specific  data,  then,  after  the  clean-up  handlers have been executed, the corresponding
       destructor functions are called, in an unspecified order.
       When a thread terminates, process-shared resources (e.g., mutexes, condition variables, semaphores,
       and file descriptors) are not released, and functions registered using atexit(3) are not called.
       After the last thread in a process terminates, the process terminates as by calling exit(3) with an
       exit status of zero; thus, process-shared resources are released  and  functions  registered  using
       atexit(3) are called.
 

Q2     If main creats two thread and exits will thread stay? or if main creats T1 and T1 creats T2 and  T1 dies What will be the fate to T2 then ?
Ans: Exiting of main() forces all child threads to kill as process cleans up .
But it can be avoided by putting
pthread_exit(NULL); in main this will enable threads to sustain even after main .
One more way is to JOIN all the treads created : it also blocks Father to wait child thread die . Main use of join call is to get status of thread after Execution
void * status;
pthread_join(threads[t] ,&status); this has to be call for each thread created .

Q3     Do pthread have PId ? Realation b/w TID and PID ?
Ans :     Yes its LWP-id they have can be viewed by

ps -eLF    // -L option is to Show threads, possibly with LWP and NLWP columns
ps -emf   // -m is for Show threads after processes

user$ ps -eLf | grep 5708    //5708 is pid of main thread which spawns two thread.
UID        PID  PPID   LWP  C NLWP STIME TTY          TIME CMD
user    5708 11297  5708  0    3 13:28 pts/2    00:00:00 ./a.out
user    5708 11297  5709  0    3 13:28 pts/2    00:00:00 ./a.out
user    5708 11297  5710  0    3 13:28 pts/2    00:00:00 ./a.out
user    5726 10354  5726  0    1 13:29 pts/1    00:00:00 grep --color=auto 5708

ALL  threads  share PID ( getpid())of parent. But LWP id will be different.
 In a single-threaded process, the thread ID is  equal to the process ID (PID, as returned by getpid(2)).  In a multithreaded process,  all  threads  have  the  same  PID,  but  each  one has a unique TID
To get tid use  : pid_t        gettid(void);
 pid_t tid;
tid = syscall(SYS_gettid);
POSIX thread id (its different from TID )
 pthread_t     pthread_self();
long tid     =   pthread_self();
note:for printing use %lu as its big on in %ld it might be NEGATIVE
From MAN page : 
 In the Linux kernel, a kernel-scheduled thread  is  not  a  distinct  construct  from  a  process.
       Instead, a thread is simply a process that is created using the Linux-unique clone(2) system call;
       other routines such as the portable pthread_create(3) call are implemented using clone(2).  Before  Linux  2.4,  a  thread was just a special case of a process, and as a consequence one thread could not wait on the children of another thread, even when the latter belongs to the same thread group.  However,  POSIX  prescribes  such  functionality, and since Linux 2.4 a thread can, and by default will, wait on children of other threads in the same thread group.
 The following Linux-specific options are for use with children created using clone(2); they cannot
   be used with waitid():
    __WCLONE
              Wait  for  "clone"  children only.  If omitted then wait for "non-clone" children only.  (A
              "clone" child is one which delivers no signal, or a signal other than SIGCHLD to its parent
              upon termination.)  This option is ignored if __WALL is also specified.
       __WALL (since Linux 2.4)
              Wait for all children, regardless of type ("clone" or "non-clone").
       __WNOTHREAD (since Linux 2.4)
              Do  not  wait for children of other threads in the same thread group.  This was the default
              before Linux 2.4.

Q4    What Zombie Process , Orphan Process, Defunct Process (in ps -eaf )
Ans : Zombie and defunct are same it is is a process that has completed execution but still has an entry in the process table. Reason of the entry is parent didn't call wait() or waitpid() for child these are the calls Required to get status of child process and removes entry form Process table .
Absence of these causes a child process to become Zombie

An orphan process is a process that is still executing, but whose parent has died. They do not become zombie processes; instead, they are adopted by init (process ID 1), which waits on its children.

Q5 Debugging with GDB with Threads ?
Ans : compile with -g flag . that will be helpful
get pid by: ps -eaf | grep exename
run gdb -p pid

Reading symbols from /lib/tls/libpthread.so.0...(no debugging symbols found)...done.
[Thread debugging using libthread_db enabled]
[New Thread -1218548928 (LWP 11719)]
[New Thread -1218552912 (LWP 11721)]

(gdb) info threads
2 Thread -1218552912 (LWP 11721) 0x08048559 in PrintHello ()
1 Thread -1218548928 (LWP 11719) 0x00d0ad58 in pthread_join () from /lib/tls/libpthread.so.0

(gdb) t 2
[Switching to thread 2 (Thread -1218540624 (LWP 18808))]#0  0x08048552 in PrintHello (threadid=0x0)
at thread.c:18
18              for(u=0;u<10000;u++)
(gdb) bt
#0  0x08048552 in PrintHello (threadid=0x0) at thread.c:18
#1  0x00407dd8 in start_thread () from /lib/tls/libpthread.so.0
#2  0x001edd1a in clone () from /lib/tls/libc.so.6

putting BreakPoint 
(gdb)break linespec thread threadno

Q5 tracking threads with PS -eLF and strace.
 gdb uses ptrace only 
-----------------------------------------------
 compiling   "gcc thread.c -pthread"

-------------
Code :


---------------Output ---
case 1 :code as above
In main: creating thread 0
Hello World! It's me, thread #0!
done thread #0!
In main: creating thread 1
Hello World! It's me, thread #1!
done thread #1!
--------main is leaving
------------------------------
case 2 commenting these two

in main function
In main: creating thread 0
Hello World! It's me, thread #0!
In main: creating thread 1
Hello World! It's me, thread #1!
--------main is leaving
main left
----------------------------
------O/P--------------------
case 3 : putting just this in main

------------------------------------
In main: creating thread 0
Hello World! It's me, thread #0!
In main: creating thread 1
Hello World! It's me, thread #1!
--------main is leaving
done thread #0!
done thread #1!
Note : thread were workiing even after main thats Magic of pthread_exit(NULL) in main

Links to refer
https://computing.llnl.gov/tutorials/pthreads/#Compiling
http://www.yolinux.com/TUTORIALS/LinuxTutorialPosixThreads.html

Implementing STATE MACHINE in C

Moore Machine: A state machine which uses only Entry Actions, so that its output depends on the state, is called a Moore model. A finite state machine which produces an output for each transition.

Melay MAchine : A state machine which uses only Input Actions, so that the output depends on the state and also on inputs, is called a Mealy model. A Mealy machine has its output depend on both input and state.Shown in diagram by drawing the input/output on the edge from state 1 --->state 2

We often use mixed model.

It can be implemented in two ways

1) Nested Switch and gotos
2) Matrix for state transition Table : data of Matrix caould be Structure having new state and pointer to Action_Function (which needs to be executed )





Refer :http://www.gedan.net/2008/09/08/finite-state-machine-matrix-style-c-implementation/
http://smc.sourceforge.net/
http://sourceforge.net/projects/smc/

Semaphore and Mutex RE-Discussion

A mutex is actually a semaphore with value 1
This is Totally wrong

Semaphore : Its simple counters that indicate the status of a resource . Counter is managed by Kernel and can be accessed by API
Any thread can decrement the count to lock the semaphore (this is also called waiting on the semaphore)
Unlike mutexes, it is possible for a thread that never waited for (locked) the semaphore to post (unlock) the semaphore. This could cause unpredictable application behavior. We should avoid this if possible.

// Wait Getting resource 
W(s) 
while  (s<=0)    {
//do  nothing 
}
s=s-1;

API :sem_wait(&sem_name);   // proto  
Funtioning : If the value of the semaphore is negative, the calling process blocks; one of the blocked processes wakes up when another process calls sem_post.

//Signal Releasing Resource 
P(s)
s=s+1;

API sem_post(&sem_name)
Functioning: It increments the value of the semaphore and wakes up a blocked process waiting on the semaphore, if any

API to create  Unnamed Semaphore 
int sem_init(sem_t *sem, int pshared, unsigned int value)
API for Named 
sem_t *sem_open(const char *name, int oflag, ...);

Counting Semaphore :used where Resources are more as memory pool , socket pool.  
Binary semaphore : count = 1  

POSIX semaphore - two type :  

a.    NAMED semaphore : A named semaphore is identified by a name of the form /somename. Two processes can operate on the same named semaphore by passing the same name to sem_open. This implies that these semaphores, like System V, are system-wide and limited to the number that can be active at any one time.
The advantage of named semaphores is that they provide synchronization between unrelated process and related process as well as between threads.
The sem_open(3) function creates a new named semaphore(if it doesn,t Exists) or opens an existing named semaphore. After the semaphore has been opened, it can be operated on using 
sem_post() and sem_wait().
  When a process has finished using the semaphore, it can use
sem_close() to close the semaphore. When all processes have finished using the semaphore, it can be removed from the System using sem_unlink().

b.   UNNAMED Semaphore /Memory Semaphore :
An unnamed semaphore does not have a name. Instead the semaphore is placed in a region of memory that is shared between multiple threads (a thread-shared semaphore) or processes (a process-shared semaphore). A thread-shared semaphore is placed in an area of memory shared between by the threads of a process, for example, a global variable.
A process-shared semaphore must be placed in a shared memory region (e.g., a System V shared memory segment created using semget(2), or a POSIX shared memory object built created using shm_open(3)).
It is used between Related process like forked one. They are not available system wide. sem_open call is not required .
sem_t semid;
int sem_init(sem_t *sem, int pshared, unsigned value);
Working :  if pshared has the value 0, then the semaphore is shared between the threads of a process, and should be located at some address that is visible to all threads (e.g., a global variable, or a variable allocated dynamically on the heap).    If pshared is non-zero, then the semaphore is shared between processes, and should be located in a region of shared memory (see shm_open(3), mmap(2), and shmget(2)). (Since a child created by fork(2) inherits its parent's memory mappings, it can also access the semaphore.) Any process that can access the shared memory region can operate on the semaphore using sem_post(3), sem_wait(3), etc. Initialising a semaphore that has already been initialised results in undefined behaviour Before being used, an unnamed semaphore must be initialised using sem_init(3).
It can then be operated on using sem_post(3) and sem_wait(3).
When the semaphore is no longer required, and before the memory in which it is located is deallocated, the semaphore should be destroyed using sem_destroy(3).  
-----------------------------------------------------------------------------------------------------------------------------
Mutex: The mutex is similar to the principles of the binary semaphore with one significant difference: the principle of ownership. When a task(Thread) locks (acquires) a mutex ONLY it can unlock (release) it. If a task tries to unlock a mutex it hasn’t locked (thus doesn’t own) then an error condition is encountered and, most importantly, the mutex is not unlocked.  
Anytime a global resource is accessed by more than one thread the resource should have a Mutex associated with it. One can apply a mutex to protect a segment of memory ("critical region") from other threads.
Mutexes can be applied only to threads in a single process and do not work between processes as do semaphores.

Pros : Problem of Accidental Release Can't happen
Uses : Mutexes are used to prevent data inconsistencies due to race conditions. A race condition often occurs when two or more threads need to perform operations on the same memory area, but the results of computations depends on the order in which these operations are performed. Mutexes are used for serializing shared resources.


How to avoid Death Detection of thread who has locked Mutex??
pthread_mutex_trylock()--will attempt to lock a mutex. However, if the mutex is already locked, the routine will return immediately with a "busy" error code

PS: All above Information is for POSIX - semctl() and semop() system calls are in SYSTEM V which is bit clumsy to use . and heavy also .
ipc -s will not show posix semaphore This command is for System V

Refer this link
http://www.feabhas.com/blog/2009/09/mutex-vs-semaphores-part-1-semaphores.html http://linuxdevcenter.com/pub/a/linux/2007/05/24/semaphores-in-linux.html http://linux.die.net/man/7/sem_overview Process memory -
http://virtualthreads.blogspot.com/2006/02/understanding-memory-usage-on-linux.html  

OPEN Question what is sem_trywait pthread_mutex_trylock()

----------------------------------------------------------------

Consumer Producer Problem two thread one writes in buffer and one reads (removes ) form the same , Idea is writer should not write if its full , It should wait til some data is freed Reader should not read if tis Empty but wait till it gets fill
Algorithm   :
semaphore fillCount = 0
semaphore emptyCount = BUFFER_SIZE
procedure producer() {
     while (true) {
         item = produceItem()
         down(emptyCount)  
         putItemIntoBuffer(item)
         up(fillCount)
       }
}
procedure consumer() {
     while (true) {
        down(fillCount) item = removeItemFromBuffer()
        up(emptyCount)
        consumeItem(item)
       }
}
Code :
Links
http://en.wikipedia.org/wiki/Producer-consumer_problem
http://linuxdevcenter.com/pub/a/linux/2007/05/24/semaphores-in-linux.html?page=6

How to Run and Get status of Script From C-code

Running is pretty easy by using system() call

ret=system("./amit.sh");
printf("status is %d \n",ret/256);

-------Script--------
1 #!/bin/bash
2 echo HI;
3 echo AMIT;
4 exit 15
------------------------
Output is 15

Why 256 is required to divide?
reason when we fork a child it return status in 2 bytes status can e checked by wait() in parent . if wait is not there it will be Zombie ?
now when child goes it returns status and semd SINGNAL SIGCHLD to parent
Status is return in two bytes 1st significant is for return value 2nd byte should have all Zero in success.
so if child is returning 3 it will be 00000011|00000000 to display it as 3 we need to right shift 8 times or devide by 256 same ......

The function exit(status) causes the executable to return "status" as the return code for main(). When exit(status) is called by a child process, it allows the parent process to examine the terminating status of the child (if it terminates first).

NOTE: exit() vs _exit(): The C library function exit() calls the kernel system call _exit() internally. The kernel system call _exit() will cause the kernel to close descriptors, free memory, and perform the kernel terminating process clean-up. The C library function exit() call will flush I/O buffers and perform aditional clean-up before calling _exit()