Project1 COP5615 Operating System Principles Summer 2004 Project 2 Date Assigned: June 3rd, 2004 Date Due: Midnight, June 20th, 2004 The purpose of this project is for you to learn Java Socket abstraction in TCP/IP communication. Overview In this project, we extend our first project into a distributed version. We will distribute customers and workers in different computers. The coordinator listens on a well-known port (Server) to accept the connections from customers and workers (Client). Customers and workers use their connections to communicate with the coordinator. The coordinator accepts tasks from customers and dispatches them to available (registered) workers. The coordinator controls two kinds of buffers (shared memory) in the system same as project1. After a customer sends a task to the coordinator, it waits for the coordinator’s response. After a worker registers itself to the coordinator as a free worker, it waits for the coordinator's assignment. When the coordinator assigns a new task to a worker, the worker executes the task. After it has completed this task, it registers itself again to the coordinator. The coordinator continually check if there is new task(s) in task buffer and if there is worker(s) in the free worker buffer. If it is the case, the coordinator assign a new task to a free worker. TCP/IP Programming using the Socket abstraction Sockets are an application program interface (or high-level communication primitive) that is built on top of the TCP/IP protocols to allow communication between remote machines on the Internet. A socket is like a file, and is used very similar to a file. The important difference is when you write data to a socket it is sent to a port on a machine to which that socket is bound. Similarly when you read from a socket, the data you get is provided from the host (sent by that machine) the socket is bound to. In the TCP/IP protocol suite there are two transport layer protocols, Transmission Control Protocol (TCP) and User Datagram Protocol (UDP). TCP provides connection-oriented, reliable, and in-order delivery of packets. UDP is an unreliable and connectionless protocol which means that there is no session established between communicating hosts. A host simply sends its packet to the receiver's known port. In this project, our choice of the transport protocol is TCP where there is a distinction at the application level between socket types for a server, ServerSocket and a client, Socket. Serializable Interface: In Java, objects implementing the Serializable Interface can be delivered through Socket. Please refer to Java Doc regarding java.io.Serializable, ObjectOutputStream, ObjectInputStream for details. In project2, Task objects returned by Help.getNewTask() implement the interface of java.io.Serializable so they can be delivered between the Server and Clients. Below, we give some sample code to illustrate how one task is passed from the coordinator to a worker.. Coordinater side: ObjectOutputStream OutObject=new ObjectOutputStream(m_oSocket.getOutputStream());//the type of m_oSocket is Socket. OutObject.writeObject(m_oTask);//the type of m_oTask is Task. Worker side: ObjectInputStream InObject=new ObjectInputStream(m_oSocket.getInputStream());////the type of m_oSocket is Socket. Object o=InObject.readObject(); if(o instanceof Task){ Task task=(Task)o; }else{ ... } Utility classes There are three Utility classes provided for you. Help.java: There are three methods in this class including Help.getString (String key, String defaultValue),Help getInt (String key, int defaultValue), and Help.getNewTask(). The first two methods for for the configuration of your system. you can use them to get the properties of the system, such as the number of workers and the corresponding host for each worker, the number of customers and the corresponding host for each customer, the port of ServerSocket for the coordinator, and the number of tasks each customer will request. All these properties are defined in the provided file system.properties. For example, if you want to get the host name on which worker1 should run, you can call Help.getString("Worker.host1","~"), and if you want to get the port number that the coordinator listening on, you can call Help.getInt("Coordinaotr.port",0). The third one (Hep.getNewTask()) is for customers to get a new computation task. It is the same as project1. Task.java: It's the task interface. All tasks implement this interface. In this project, a special task named PrimeNumberTask is provided. And we will only use this. The following is the interface. import java.io.Serializable; public interface Task extends Serializable { public abstract void execute();//Worker threads should call this method when it wants to execute the task. public abstract int getID();//it will give the identity of the task. } PrimeNumberTask.java: This is an example implementation of Task interface. When you call Help.getNewTask(), one instance of it will be returned. This sample task is to determine whether a randomly generated number is prime or not and print out the result. It is just a dummy computation provided for you to simulate the task. Note: The provided files are different from project1 and should not be modified. System Design This section is to help you understand the project. It is not necessary to follow it when implementing your own system. 1) To start the whole system, we will use a single program, named Start.java. The program spawns a thread to run as the Coordinator and remotely starts a set of processes running as customers/workers on the machines specified in the properties. 2) For coordinator: To increase the concurrency in the server side, the coordinator should spawn threads to serve concurrent requesting clients. Failure to do so will result in significant points lost. 3)For Worker. After connecting to the coordinator, it sends the registration information to the coordinator. After it receives a new task, it call the execute() method to execute the task. Then it sends the registration information again 4)For Customer After connecting to the coordinator, it sends a new task to the coordinator. After it receives the ACK message from coordinator, it sends another one task. How to Start a Remote Process To start a process on a remote machine we use the remote login facility ssh in Unix systems. It accepts the host (computer) name as a parameter and commands to execute separated by semicolons. To execute the ssh command from a Java program, you should use the exec() method of the Runtime class. First, we need to get the current directory of the user (all the files necessary for this project should be in the same directory, and Start.java will be run in this directory),Second, we invoke exec() method. Below is a sample to run the first Worker in the remote machine. String path = System.getProperty("user.dir");//get current directory of the user String executestring="ssh " + Help.getString("Worker.host0","~") + " cd " + path + "; rm -f Worker0; java Worker > Worker0"; Runtime.getRuntime().exec(executestring); Here ssh is the command to start a process that will run on the host given by Help.getString("Worker.host0","~"). Immediately after the host name you need to specify the cd command to make the current working directory of the new remote process be the same with the starting program. Hence the output of the remote processes will all be directed to the same directory where we originally started the system. The path is another program variable which specifies the full path name of the directory where we invoked the Start.java program. Note that you should use this path as exactly returned by the Java method. You need not append anything to this string. rm-f Worker0 means to delete the file named Worker0.After the semicolon you should specify the name of the program to run on the remote machine. The following ">" redirects the standard output of the program to the file named Worker0, which is the log (result) file for the Client. Configuration File: The configuration of the system will be provided by a text file named system.properties. An example of this file is as follows: #Coordinator Coordinator.host=sand.cise.ufl.edu Coordinator.port=12345 Coordinator.task_buffersize=2 #Worker Worker.num=2 Worker.host0=sand.cise.ufl.edu Worker.host1=sand.cise.ufl.edu #Customer Customer.num=3 Customer.task_num=2 Customer.host0=sand.cise.ufl.edu Customer.host1=sand.cise.ufl.edu Customer.host2=sand.cise.ufl.edu This is just an example of a configuration. You can change the hosts/port to avoid conflicts with other students while you are developing your code. Before turning in your project, you should test it with different property values. Please note that Coordinator.host should be the name of the host where you run Start.java . Input and Output of the System All outputs should be directed to standard output (terminal). Please note the ssh command remotely invoking Workers/Customers issued by Start.java will redirect the standard output to a file. The sample output files are provided(Coodinator,Work0,Work1,Customer0,Customer1,Customer2), please note that the Coodinator prints outputs to the standard output, for your convenience, it is named as Coordinator . You can download all files provided here. Requirements System configuration should be read from system.properties file. Parameters in this file should not be hardcoded into the source code. All programs and threads should terminate gracefully after completing their tasks. No run-away processes is allowed to exist. Please visit the project FAQ page for downloading the small script used by us for cleaning run-away processes. Please modify it to suit your needs or write your own scripts. Thanks. The whole system should be started with a program named Start.java. Only the final result (see the sample file) of client and server should be printed. You should not submit anything other than makefile, report and java files. The requirements in Project Overview and Requirements document should be followed strictly. Submission Please follow the submission guidelines in Project Overview and Requirements document . Resources You may find tutorials and sample code on general Java programming, TCP/IP programming in Java and Threads by following the link http://java.sun.com and searching for specific information you need. You may also find many other relevant resources by searching the web.