User Tools

Site Tools


nnm:cloud_computing

This is an old revision of the document!


nextnano.cloud

Useful HTCondor commands for the Command Prompt

  • condor_submit <filename>.sub Submit a job to the pool.
  • condor_q Shows current state of own jobs in the queue.
    • condor_q -nobatch -global -allusers Shows state of all jobs in the cluster. Of all users.
    • condor_q -goodput -global -allusers Shows state and occupied CPU of all jobs in the cluster.
    • condor_q -allusers -global -analyze Detailed information for every job in the cluster.
    • condor_q -global -allusers -hold Shows why jobs are in hold state.
  • condor_status Shows state of all available resources.
  • condor_status -long Shows state of all available resources and many other information.
  • condor_status -debug Shows state of all available resources and some additional information, e.g. WARNING: Saw slow DNS query, which may impact entire system: getaddrinfo(<Computername>) took 11.083566 seconds.
  • condor_rm Remove jobs from a queue:
    • condor_rm -all Removes all jobs from a queue.
    • condor_rm <cluster>.<id> Removes jobs on cluster <cluster> with id <id> (It seems <cluster>. can be omitted, and id is the JOB_IDS number.)
  • condor_release -all If any jobs are in state hold, use this command to restart them.
  • condor_restart Restart all HTCondor daemons/services after changes in config file.
  • condor_version Returns the version number of HTCondor
  • condor_store_cred query Returns info about the credentials stored for HTCondor jobs
  • condor_history Lists the recently submitted jobs. If for a specific job ID the status has the value ST=C, then this job has been completed (C) successfully.
  • condor_status -master: returns Name, HTCondor Version, CPU and Memory of central manager
  • Open Command Prompt cmd.exe as Administrator. Type in: net start condor. This has the same effect as restarting your computer, i.e. the networking service condor is started. This is useful if you have changed your local condor_config file.

Configuration options for the Central Manager computer

With this option in the condor.config file on the central manager, one can set a policy that the jobs are spread out over several machines rather than filling all slots of one computer before filling the slots of the other computers.

##------nn: SPREAD JOBS BREADTH-FIRST OVER SERVERS
##-- Jobs are "spread out" as much as possible,
##   so that each machine is running the fewest number of jobs.
NEGOTIATOR_PRE_JOB_RANK = isUndefined(RemoteOwner) * (- SlotId)

FAQ

Q: I submitted a job to HTCondor, but nothing happens. nextnanomat says “transmitted”.

A: It could be that nextnanomat does not have read in all required settings. You can try to type in the command line condor_restart. Please make sure that you entered your credentials using condor_store_cred add -debug. You should then start nextnanomat again.

Q: I submitted a job to HTCondor, but the Batch line of nextnanomat is stuck with preparing. What is wrong?

A1: Did you store your credentials after the installation of HTCondor? If not, enter condor_store_cred add into the command prompt to add your password, see above (Recommended Installation Process).

A2: Did you change your password recently? If yes you have to reenter your credentials for HTCondor. Enter condor_store_cred add into the command prompt to add your password, see above (Recommended Installation Process). If this does not work, try to enter condor_store_cred add -debug for more output information on the error.

Q: I specified target machines in Tools - Options. Afterwards every submitted job to HTCondor is stuck with transmitting. What is wrong?

A: The value for UID_DOMAIN within the condor_config file needs to be the same for every computer of your cluster. (You can easily test it in a command prompt with condor_status -af uiddomain) If it's not the same value, no matching computer will be found and the job won't be transmitted successfully.

Problems with HTCondor

Error: communication error

If you receive the following error when you type in condor_status

C:\Users\"<your user name>">condor_status
Error: communication error
CEDAR:6001:Failed to connect to <123.456.789.123>

you can check whether the computer associated with this IP address is your HTCondor computer using the following command.

nslookup 123.456.789.123

It is also a good idea to type in

nslookup

This will return the name of the Default Server that resolves DNS names. If it is not the expected computer, you can open a Command Prompt as Administrator and type in ipconfig /flushdns to flush the DNS Resolver Cache.

C:\Users\"<your user name>">ipconfig /flushdns

If the DNS address cannot be resolved correctly it could be related to a VPN connection that has configured a different default server for Domain Name to IP address mapping. E.g. if your Windows Domain is called contoso.com (which is only visible within your own network and your own HTCondor pool) but your DNS is resolved to www.contoso.com (which might be outside your local HTCondor pool).

Error: ''condor_store_cred add'' failed with ''Operation failed. Make sure your ALLOW_WRITE setting include this host.''

Solution: Edit condor_config file and add host, i.e. local computer name (here: nn-delta).

    ALLOW_WRITE = $(CONDOR_HOST), $(IP_ADDRESS)
==> ALLOW_WRITE = $(CONDOR_HOST), $(IP_ADDRESS), nn-delta

Error? Check the Log files

If you encounter any strange errors, you can find some hints in the history or Log files generated by HTCondor. You can find them here:

C:\condor\spool

  • history

C:\condor\log

  • CollectorLog
  • MasterLog
  • MatchLog
  • NegotiatorLog
  • ProcLog
  • SchedLog
  • ShadowLog
  • SharedPortLog
  • StarterLog
  • StartLog

More details can be found here: Logging in HTCondor

nnm/cloud_computing.1704295010.txt.gz · Last modified: 2024/01/03 16:16 by stefan.birner